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PREFACE 


This  report  contains  a review  o*  the  literature  relevant  to  team  training,  surveying 
and  describing  current  instructional  and  evaluative  techniques  within  the  Services.  State- 
of-the-art  gaps  were  identified  and  research  needs  documented  as  an  aid  toward  planning 
improved  research  programs  in  team  training  and  evaluation. 

This  study  was  performed  in  HumRRO’s  Eastern  Division,  Dr.  J.  Daniel  Lyons, 
Director.  The  work  was  performed  under  the  general  direction  and  supervision  of 
Dr.  Robert  J.  Seidel,  Program  Director,  Instructional  Technology  Group,  who  assisted 
the  authors  of  this  report  in  the  identification  of  potential  team  training  research  areas. 
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SUMMARY 


BACKGROUND 

This  review  of  team  training  and  evaluation  was  undertaken  by  HumRRO  in  order  to 
provide  an  information  base  that  the  Defense  Advanced  Research  Projects  Agency  could 
use  as  a foundation  to  facilitate  decisions  regarding  future  research  program  support. 

In  order  to  pursue  their  mission  of  operational  readiness,  the  Services  conduct  most 
of  their  training  in  the  operational  commands.  In  the  past,  most  training  research  has 
been  focused  on  individual  training  at  schools  and  at  other  institutional  locations.  Con- 
sidering the  amount  of  team  training  conducted  by  the  Services,  either  formally  recognized 
as  training  or  combined  with  operations,  the  funds  committed  to  R&D  support  of  such 
training  are  relatively  small.  There  is  a need,  therefore,  for  greater  research  emphasis  to 
increase  the  effectiveness  of  team  training  and  evaluation. 


PURPOSE 

The  purpose  of  this  literature  review  was  to  provide  information  that  would  be  useful 
for  planning  research  and  development  programs  in  the  area  of  team  training.  Such  pro- 
grams are  needed  to  develop  improved  team  training  methods  and  technologies,  as  well  as 
to  provide  new  measurement  techniques  and  procedures  for  evaluation.  The  specific 
objectives  in  preparing  this  review  were  to  address  the  following  questions: 

• What  state-of-the-art  gaps  are  there  in  team  training  strategies  and  evaluation 
techniques? 

• What  new  team  training  strategies  appear  to  hold  promise  for  application  to 
the  DoD  environment? 

• What  new  evaluation  techniques  can  be  used  to  assess  team  training? 


METHOD 

A thorough  search  was  made  of  the  major  documentation  sources  for  publications 
that  would  be  relevant  to  team  training  and  evaluation.  The  initial  search  included  the 
Educational  Resources  Information  Center  (ERIC),  National  Technical  Information  Ser- 
vice (NTIS),  and  Defense  Documentation  Center  (DDC)  indices.  In  addition  to  DoD- 
sponsored  research  reports,  the  survey  included  publications  from  the  social  psychological 
areas,  such  as  group  dynamics  and  the  industrial  training  field.  Secondary  sources  and 
cross-references  were  then  identified  and  reviewed. 

HumRRO  personnel  with  experience  in  the  team  training  area  were  consulted  for 
information  about  current  military  team  training  methods.  The  most  recent  military 
doctrine  regarding  team  training  procedures  was  obtained  and  reviewed.  Military  per- 
sonnel were  contacted  in  all  the  Services  to  identify  team  training  approaches  now  in 
operation.  Current  team  training  and  evaluation  studies  under  way  within  the  Army 
Research  Institute  and  the  other  Services  were  examined  and  described. 


FINDINGS  AND  IMPLICATIONS 

As  an  aid  toward  organizing  and  analyzing  the  team  training  information  obtained, 
a classification  scheme  was  used  to  categorize  the  training  techniques  and  situations 
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discussed  in  this  review  along  two  dimensions.  On  one  dimension,  training  focus,  a distinction 
was  made  between  “team”  training  and  “multi-individual”  training.  Multi-individual  training 
occurs  in  a group  context  but  focuses  on  the  development  of  individual  skills.  Team  training, 
on  the  other  hand,  is  focused  on  developing  team  skills  such  as  coordination  and  cooperation. 
The  type  of  task  situation  was  the  second  dimension  used  to  classify  the  training  techniques 
reviewed.  Task  situations  were  categorized  as  either  “established”  or  “emergent.”  Estab- 
lished situations  are  those  in  which  the  tasks  and  the  activities  required  to  perform  these 
tasks  can  be  almost  completely  specified.  Emergent  situations  are  those  in  which  all  tasks  and 
activities  cannot  be  specified  and  the  probable  consequences  of  certain  actions  cannot  be  pre- 
dicted. This  type  of  situation  allows  for  unanticipated  behaviors  to  emerge. 

Team  training  studies  and  practices  were  categorized  according  to  the  classification 
scheme  described.  These  studies  followed  two  conceptual  models  of  team  behavior 
Stimulus-Response  (S-R)  and  Organismic.  The  S-R  model  adherents  tended  to  study 
team  training  in  laboratory  settings  derived  from  established  task  situations.  More  real- 
istic environments  were  used  by  other  researchers  who  attended  to  emergent  factors  in 
the  job  situation  (the  organismic  approach).  It  was  this  latter  group  of  investigators  who 
demonstrated  the  need  for  training  in  team  skills,  even  though  individual  skill  proficiency 
was  found  to  be  a prerequisite  for  effective  team  training  and  performance.  Other  con- 
clusions which  were  drawn  from  the  literature  are: 

• The  team  context  is  not  the  proper  location  for  initial  individual  skill 
acquisition. 

• Performance  feedback  is  critical  to  the  learning  of  team  as  well  as  indi- 
vidual skills. 

Current  military  team  training  practices  were  described  as  varying  in  cost,  in  degree 
of  fidelity  to  the  operational  site,  and  in  the  variety  and  complexity  of  the  tasks  trained. 
Subjective  methods  of  evaluation  were  found  to  be  predominant. 

Evaluation  of  team  training  effectiveness  depends  upon  the  development  of  objective 
team  performance  measurement  instruments  and  procedures.  This  review  of  the  literature 
and  current  practice  supports  the  conclusion  that  this  area  contains  many  problems  that 
must  be  solved  before  substantial  improvement  in  team  training  evaluation  can  occur. 

The  production  of  standardizable,  replicable  test  conditions  and  the  establishment  of 
accepted  performance  criteria  are  two  such  problems. 

A phased  research  program  was  suggested  as  warranting  support  in  order  to  over- 
come gaps  in  the  current  state  of  knowledge.  The  major  factor  in  this  program  is  the 
need  for  methods  that  differentiate  individual,  multi-individual,  and  team  skills  and  train- 
ing requirements.  Such  techniques,  which  could  be  integrated  with  job/task  analyses, 
would  discriminate  between  isolated  and  interactive  behaviors.  Possible  techniques  to 
study  for  this  purpose  are  variations  of  interaction  analyses  and  other  unobtrusive  meas- 
ures. Successful  development  of  this  methodology  will  provide  an  opportunity  to 
establish  team  performance  standards  criteria  as  part  of  training  requirements  analyses. 
Development  of  these  techniques  would  then  permit  other  research  studies  to  be  accom- 
plished effectively. 

Issues  identified  in  this  review  which  merit  research  support  include  the  following: 

• Team  Feedback 

What  changes  in  training  methods  or  content  result  from  feedback?  Which 
method  of  providing  team  feedback  is  most  useful  (e.g.,  video  recording 
versus  post-exercise  debriefing)? 


* 


vl 


; 

t 

r 


l 


1 


[ 


• Assessment  Training 

Will  training  of  team  members  to  assess  their  own  performance  result  in 
greater  team  proficiency  than  if  no  self-assessment  training  is  provided? 

Will  training  to  assess  the  performance  of  other  team  members  affect 
proficiency. 

• Simulation  Fidelity 

What  degree  of  simulation  fidelity  is  critical  for  effective  team  training?  Can 
two-sided  engagement  simulation  techniques  be  applied  to  the  training  of 
team  skills?  Although  low-fidelity  simulations  were  found  to  be  ineffective 
in  established  task  situations,  would  they  be  more  useful  in  emergent 
situations? 

• Team  Composition/Structure 

What  are  the  effects  of  degree  of  team  member  familiarity  on  the  acquisition 
of  team  skills?  Will  the  length  of  time  that  a team  functions  together  effect 
proficiency? 


• Skill  Training  Sequencing 

Is  it  more  efficient  to  train  individual  skills  first  and  then  team  skills,  or 
team  skills  first,  or  both  concurrently? 

Combined  studies  dealing  with  more  than  one  of  these  research  issues  are  feasible. 

For  example,  feedback  or  skill  sequencing  issues  can  be  studied  in  environments  that  vary 
in  their  degree  of  simulation  fidelity,  or  in  task  situations  that  are  established  or  emergent. 

The  studies  that  have  been  suggested  point  out  the  considerable  research  and  devel- 
opment effort  needed  in  the  team  training  and  evaluation  area.  With  further  support,  our 
state  of  knowledge  in  the  area  should  expand  considerably. 
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Chapter  1 

INTRODUCTION 

Interdependent,  coordinated  team  performance  is  a predominant  characteristic  of 
most  operational  activities  within  the  military  Services.  Training  of  teams  in  operational 
units  is  the  transition  between  initial  individual  training  and  combat.  The  influence  of 
effective  team  behavior  upon  system  performance  clearly  has  implications  for  training. 
Team  training  is  more  difficult,  and  is  assumed  to  be  more  costly,  than  is  individual  train- 
ing. It  is  usually  performed  in  operational  environments  or  in  high-fidelity  simulations  of 
such  environments.  For  these  reasons,  sound  data  are  needed  for  use  in  determining  the 
relative  effectiveness  of  alternate  training  concepts  as  a means  for  improving  team  training. 


BACKGROUND 

In  pursuing  their  mission  of  operational  readiness,  the  Services  conduct  most  (up  to 
90%)  of  their  training  in  the  operational  commands.  However,  in  the  past,  most  training 
research  has  been  focused  on  individual  training  at  Schools  and  at  other  institutional 
locations. 

Maj.  Gen.  Paul  F.  Gorman,  Deputy  Chief  of  Staff  for  Training,  U.S.  Army  Training 
and  Doctrine  Command,  recently  stated  [93]  that  there  is  a need  for  increased  research 
emphasis  on  what  is  called  “collective  training”  (team  or  crew  tactical  training  in  a unit 
context).  A recent  Defense  Science  Board  task  force  report  [26]  stated  that  in  view  of 
the  amount  of  team  training  that  goes  on  in  the  Services,  either  formally  recognized  as 
training  or  combined  with  operations,  the  funds  committed  to  R&D  support  for  team 
training  are  relatively  small.  In  Fiscal  Year  1974,  the  Army  Research  Institute  (ARI) 
conducted  the  largest  program  of  unit  research  in  history,  and  yet  spent  less  than  $2 
million— only  11%  of  the  human  resources  research  program  in  that  year.  Most  of  the 
money  in  that  program  went  for  improved  institutional  training  or  exploration  of  other 
individual  training  issues. 

PURPOSE  OF  STUDY 

This  review  of  the  team  training  area  was  undertaken  to  provide  an  information  base 
that  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  and  the  Services  could 
use  to  facilitate  planning  and  decisions  regarding  future  research  program  support.  Team 
training  research  programs  are  to  develop  better  team  training  methods  and  technologies 
and  to  provide  new  measurement  techniques  and  procedures  for  team  evaluation.  The 
following  objectives  were  established  to  guide  the  preparation  of  this  review: 

1.  Describe  existing  instructional  strategies  and  evaluation  techniques  relevant 
to  team  training. 

2.  Identify  state-of-the-art  gaps  in  team  training  and  evaluation  and  suggest 
new  team  training  strategies  and  evaluation  techniques  which  warrant 
further  study. 


APPROACH 

Review  of  Technical  Literature 

An  extensive  search  was  made  of  the  major  documentation  sources  in  order  to  amass 
an  initial  listing  of  publications  relevant  to  team  training.  The  emphasis  was  on  a docu- 
ment’s applicability  to  tactical  training  of  teams.  However,  instructional  strategies  and 
evaluation  techniques  that  had  been  used  in  other  contexts  also  were  examined  for  possi- 
ble relevancy. 

The  following  major  sources  were  searched: 

Educational  Resources  Information  Center  (ERIC).  These  indices  were  searched 
for  relevant  titles  over  the  preceding  10-year  period.  The  descriptors  used  in  this  search 
included  team  training,  military  training,  industrial  training,  group  dynamics,  group  effec- 
tiveness, group  instruction,  small  group  instruction,  training  techniques. 

Defense  Documentation  Center  (DDC).  The  indices  of  the  preceding  10-year 
period  were  searched  using  the  following  descriptors:  team  training,  group  dynamics, 
group  problem  solving,  training  crews. 

Psychological  Abstracts.  The  descriptors  used  to  search  these  indices  were  group 
dynamics,  training,  training/industrial,  training/military,  and  group/small.  Here  too,  a 10- 
year  period  was  selected  for  the  in-depth  search. 

National  Technical  Information  Service  (NTIS).  The  general  area  of  “military 
sciences”  was  canvassed  for  appropriate  literature.  Also,  team  training,  group  dynamics, 
group  problem  solving,  and  group  training  were  used  as  descriptors. 

HumRRO  Library.  This  library  specializes  in  publications  that  are  relevant  to 
military  training  and  evaluation.  A sizable  number  of  documents  were  thus  obtained  “in- 
house”  for  this  review. 

The  initial  list  contained  many  hundreds  of  titles.  Each  document’s  abstract  was 
reviewed  to  ensure  that  only  those  publications  which  were  relevant  to  team  training 
and  evaluation  would  be  selected  for  the  literature  survey.  The  documents  judged  to  be 
relevant  were  then  obtained  and  reviewed. 

Once  the  search  in  the  initial  sources  was  completed,  secondary  sources  (e.g..  Journal 
of  Applied  Psychology,  Training  in  Business  and  Industry)  were  canvassed  for  relevant 
articles.  In  addition,  “key”  documents  that  were  more  than  10  years  old  were  identified 
from  the  cross-references  in  documents  already  reviewed.  These  key”  documents  were 
also  obtained  and  reviewed.  All  these  items  are  listed  in  the  References  section  of 
this  report. 

Sample  of  Current  Military  Team  Training  Techniques 

The  most  recent  military  doctrine  regarding  team  training  procedures  was  obtained 
and  reviewed.  Current  studies  under  way  within  the  Army  Research  Institute  and  the 
other  Services  were  examined.  Military  personnel  were  contacted  in  all  the  Services  to 
identify  team  training  approaches  currently  in  operation.  HumRRO  personnel  with 
experience  in  the  team  training  area  (e.g.,  Dr.  Joseph  Olmstead)  were  consulted  for  infor- 
mation about  current  military  team  training  methods.  Examples  of  current  techniques 
were  then  chosen  for  inclusion  in  this  report  and  are  described  in  Chapter  4. 


DEFINITIONS 

As  can  be  seen  from  the  variety  of  sources  chosen  for  the  initial  literature  search, 
HumRRO  reviewers  had  not  yet  made  a clear  conceptual  delineation  of  the  team  training 
area.  They  viewed  team  training  research  as  part  of  the  more  general  subject  matter  of 
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small  group  research  or  group  dynamics.  This  larger  field  had  generated  literally  thousands 
of  empirical  studies  and  theoretical  papers,  most  of  which  were  only  tangentially  relevant 
to  the  sub-area  of  team  training.  An  understanding  was  needed  of  what  makes  the  team 
context  unique. 

Before  a state-of-the-art  review  on  team  training  could  be  accomplished,  one  needed 
to  define  what  constitutes  a “team.”  In  various  circumstances,  teams  may  be  made  up  of 
two  or  more  individuals  (with  no  upper  limit)  and  they  may  be  characterized  by  virtually 
any  degree  of  formal  organizational  structure  and  permanency  [49] .’  Rather  than 
attempting  to  develop  one  more  definition,  the  authors  viewed  the  description  given  by 
Glaser,  Klaus,  and  Egerman  [40]  as  providing  the  most  meaningful  distinction  between 
small  groups  and  teams  considered  in  this  review: 

• “Teams”  have  the  following  characteristics: 

1.  They  are  relatively  rigid  in  structure,  organization,  and 
communication  pattern. 

2.  The  task  of  each  team  member  is  well  defined. 

3.  The  functioning  of  the  team  depends  upon  the  coordinated 
participation  of  all  or  several  individuals. 

• In  contrast,  “small  groups”  differ  in  that  they  generally: 

1.  Have  an  indefinite  or  loose  structure,  organization,  and 
communication  pattern. 

2.  Have  assignments  which  are  assumed  in  the  course  of  group 
interaction  rather  than  designated  beforehand. 

3.  The  group  product  can  be  a function  of  one  or  more  of  the 
group  members  involved  depending  upon  the  quality  and 
quantity  of  their  participation.  (Glaser,  Klaus,  Egerman  [40]) 

These  definitions  were  used  to  guide  the  final  selection  of  studies  and  papers  for  in- 
depth  review. 

Instructional  strategies  and  evaluation  techniques  for  “teams”  should  take  these  dimen- 
sions into  consideration.  In  addition,  teams  are  goal-  or  mission-oriented  [49]  and  so  the 
specific  context  in  which  the  team  will  operate  must  be  considered  before  any  training  or 
evaluation  technique  can  be  applied. 

The  question  of  whether  “team”  or  “multi-individual”  behaviors  are  involved  in  a 
given  context  has  not  been  adequately  studied.  As  derived  from  the  above  discussion, 
“team  training”  can  he  defined  as: 

• The  training  of  two  or  more  individuals  who  are  associated 
together  in  work  or  activity.  The  team  is  relatively  rigid 
in  structure  and  communication  pattern.  It  is  goal-  or 
mission-oriented  with  the  task  of  each  team  member  well- 
defined.  The  functioning  of  the  team  depends  upon  the 
coordinated  participation  of  all  or  several  individuals.  The 
focus  of  team  training  and  feedback  is  on  team  skills  (e.g., 
coordination),  activities,  and  products. 

On  the  other  hand,  “multi-individual  training”  can  be  defined  as: 

• The  training  of  two  or  more  individuals  who  are  associated 
in  a group  context.  The  trainees  may  or  may  not  be  part 


1 This  document  by  Hall  and  Rizzo  provides  an  excellent  description  of  how  definitional  problems 
have  troubled  the  team  training  field  and  of  the  various  ways  in  which  the  research  literature  has  dealt 
with  these  difficulties. 


of  a team.  However,  the  focus  of  the  training  and  feedback 
is  on  individual  skills,  activities,  and  products. 

Methods  are  needed  by  which  one  can  analyze  given  task  situations  and  derive  the 
requirements  for  individual  or  team  training. 

In  the  chapters  that  follow,  a distinction  will  be  made  on  this  dimension— that  is, 
whether  the  study  or  technique  reviewed  is  pertinent  to  multi-individual  or  to  team 
contexts. 
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Chapter  2 

REVIEW  OF  TEAM  TRAINING  RESEARCH 

Research  in  the  area  of  team  training  has  been  going  on  for  approximately  20  years, 
albeit  at  a relatively  low  level  of  effort  122,  23,  45,  61,  71,  73,  79,  80].  Despite  the 
importance  of  the  area,  Glanzer  and  Glaser  [43]  noted  in  1955  that  relatively  little  formal 
knowledge  existed  concerning  methods  of  describing  and  analyzing  team  performance. 

They  cited  the  complexity  of  team  training  problems  as  contributing  to  that  situation: 

“In  the  investigation  of  the  areas  of  team  as  opposed  to  individual  training  and  performance, 
problems  appear  of  an  entirely  new  order  of  magnitude.” 

Twenty  years  later,  the  Defense  Science  Board  described  some  of  the  difficulties  in 
performing  team  training  research  that  have  prevented  significant  breakthroughs: 

This  kind  of  R&D  must  be  piggybacked  on  operations  in  the  field, 
large  numbers  of  R&D  personnel  are  required,  the  opportunities  for 
data  collection  during  the  exercise  are  marginal,  inferential  statistics 
and  psychometrics  were  not  designed  for  this  order  of  complexity, 
there  are  limited  opportunities  for  repeated  trials,  the  ultimate  test  of 
team  training  is  combat,  which  cannot  be  simulated.  [26] 

The  difficulties  cited  in  doing  team  training  research  have  prevented  a thorough  study 
of  the  problems  and  research  needs  that  have  been  identified  during  the  intervening  years. 
However,  some  definite  progress  has  been  made  and  certain  research  trends  have  been  iden- 
tified and  described  [44]. 

Research  findings  that  have  been  reported  on  some  major  team  training  problems  and 
issues  will  be  described  in  the  eight  sections  that  follow.  The  topics  discussed  are: 

• Section  1— The  “Stimulus-Response”  and  “Organismic”  models  which  underlie 
different  approaches  to  the  study  of  team  training  [ 1 ] . 

• Section  2— The  components  of  “Emergent”  versus  “Established”  situations  and 
their  effects  on  team  training  research. 

• Section  3— The  relative  importance  and  interactions  of  individual  training  and 
team  training. 

• Section  4— Descriptions  of  and  problems  involved  in  studying  team  skills. 

• Section  5— The  degree  of  simulation  fidelity  and  its  influence  on  effective 
training. 

• Section  6— The  effects  of  feedback  on  team  performance,  and  techniques  used 
to  present  knowledge  of  results  to  team  members. 

• Section  7— Team  structure  and  composition  and  their  effects  on  team  training. 

• Section  8— The  benefits  of  a systems  approach  to  the  development  of  effec- 
tive team  training. 

1.  TWO  CONCEPTUAL  APPROACHES  TO  TEAM  TRAINING  RESEARCH 

Two  team  training  models  have  been  distinguished  by  Alexander  and  Cooperband  [1] 
on  the  basis  of  the  situations  in  which  team  behavior  takes  place.  The  variables  and 
environment  that  researchers  select  for  study  depend  upon  the  team  model  they  choose. 
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The  “Stimulus-Response”  model  is  applied  to  teams  that  operate  primarily  in 
established  situations,  in  which  the  tasks,  and  the  activities  required  to  perform  them, 
can  be  almost  completely  specified,  and  in  which  the  assignment  of  functions  among 
team  members  and  their  equipment  is  relatively  rigid.  This  viewpoint  underlies  attempts 
to  apply  certain  principles  and  techniques  of  individual  learning  (e.g.,  operant  conditioning) 
to  research  on  team  training.  Studies  performed  at  the  American  Institutes  for  Research 
Team  Training  Laboratory  are  illustrative  of  the  model.  For  example,  in  one  study  [65] 
reinforcement  of  team  response  was  manipulated  to  measure  differences  in  team  mem- 
ber proficiency. 

In  the  “Organismic”  model  [1]  the  team  is  considered  to  be  a synthetic  organism  of 
which  individuals  are  components.  This  model  is  oriented  toward  teams  operating  in 
environments  that  include  a significant  proportion  of  emergent  situations.  Although  there 
are  defined  task  assignments,  the  individual  has  a considerable  degree  of  discretion  in  how 
to  perform  the  task  under  various  contingencies.  Consequently,  proficient  team  performance 
can  depend  upon  the  development  of  appropriate  team  procedures  enabling  members  to 
cope  with  the  environment.  Thus,  adaptive  innovations  are  needed  and  decision-making 
and  problem-solving  skills  are  emphasized  [1],  An  example  of  this  viewpoint  is  the  work 
done  by  Chapman,  et  al.  [17],  in  which  teams  in  a simulated  air  defense  direction  center 
were  presented  with  sudden  disruptive  changes  in  their  environment  and  were  required  to 
react  to  the  new  stimuli  quickly  in  order  to  solve  given  problems  precisely. 
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2.  ESTABLISHED  VS.  EMERGENT  SITUATIONS 

The  two  conceptual  approaches  described  in  Section  1 take  as  their  basis  the  context 
in  which  team  behavior  occurs.  This  context  is  actually  a continuum  of  situations,  the 
end  points  of  which  are  described  as  established  or  emergent.  Boguslaw  and  Porter  [10] 
define  these  situations  as  follows: 

An  established  situation  is  one  in  which  (1)  all  action-relevant 
environmental  conditions  are  specifiable  and  predictable,  (2)  all 
action-relevant  states  of  the  system  are  specifiable  and  predictable, 
and  (3)  available  research  technology  or  records  are  adequate  to 
provide  statements  about  the  probable  consequences  of  alternative 
actions.  An  emergent  situation  is  one  in  which  (1)  all  action- 
relevant environmental  conditions  have  not  been  specified,  (2)  the 
state  of  the  system  does  not  correspond  to  relied-upon  predictions, 
and  (3)  analytic  solutions  are  not  available,  given  the  current  state 
of  analytic  technology. 

An  established  situation  was  the  primary  focus  of  a research  program  of  the  Team 
Training  Laboratory  of  the  American  Institutes  for  Research  [62],  The  advantages  of 
such  a situation  for  research  are: 

• It  is  relatively  easy  to  abstract  key  characteristics  of  real-world  team  tasks 
and  represent  them  in  the  laboratory. 

• Experiments  can  be  done  relatively  inexpensively. 

• Studies  can  be  conducted  in  reasonable  time  periods. 

• Monitoring  of  team  activities  is  feasible. 

On  the  other  hand,  simplifying  the  representation  of  team  functions  can  mask  or  omit 
possible  important  variables  which  influence  behavior  in  the  real  world.  By  abstracting 
the  situational  context  in  the  laboratory,  there  can  be  a loss  of  opportunity  for  trainees 
to  react  to  breakdowns  and  problems  that  may  arise  in  an  operational  setting. 
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Providing  skills  to  deal  with  emergent,  unstructured  situations  was  seen  as  a major 
goal  of  team  training  in  studies  by  the  System  Development  Corporation  [ 1 ] . The  devel- 
opment of  coordinative  skills  was  stressed  although  it  was  recognized  that  these  are  based 
upon  a certain  minimum  attainment  of  individual  skill  Team  training  devices  and  tech- 

iniques  were  seen  as  requiring  orientation  toward  the  training  of  innovative  behaviors  and 

skills  necessary  to  adapt  to  unforeseen  problems  [11,  25]. 

An  emergent  situation  permits  a more  realistic,  less  abstract  approach  to  the  investi- 
gation of  team  training  variables  than  does  an  established  situation.  When  a team  was 
formed  in  the  laboratory,  its  prior  motivational  characteristics  were  important  determi- 
nants of  performance.  But,  when  teams  were  studied  while  carrying  out  face-valid  activi- 
ties, prior  motivational  states  appeared  to  be  less  important.  What  was  important  in  this 
case  was  training  in  coordination  such  that  team  members  became  fully  aware  of  their 
responsibilities  to  compensate  for  the  inabilities  of  others,  or  to  overcome  temporary 
problems  when  the  situation  called  for  it  [37], 

By  viewing  task  situations  as  established  or  emergent,  one  can  classify  the  training 
techniques  and  research  studies  discussed  in  this  review  as  pertinent  to  one  type  of  con- 
text or  the  other.  Similarly,  the  distinction  between  team  training  and  multi-individual 
training  comprises  a second  classificatory  dimension.  The  resulting  2x2  matrix— Task 
Situation/Training  Focus— constitutes  a classification  scheme  which  will  be  used  in  the 
remaining  sections  of  this  chapter  and  in  later  chapters  to  categorize  the  research  studies 
and  training  procedures  surveyed.1 


3.  INDIVIDUAL  VS.  TEAM  TRAINING 

The  studies  in  this  section  were  classified  as  shown  in  Table  1.  Although  the  focus  of 
this  literature  review  was  on  team  training,  some  relevant  studies  dealt  with  individual  skills 
training  in  a multi-individual  context. 


Table  1 


Individual  vs.  Team  Training: 

Classification  of  Stvdk^  Reviewed  in  Chapter  2,  Section  3 


z 
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TRAINING  FOCUS 


Team 


Briggs  & Johnston  111,  1 3 1 
Hall  & Rizzo  [49] 

Klaus  & Glaser  [62,  63,  64] 


Multi-Individual 


Horrocks,  et  a/.  [52,  53] 


Alexander  & Cooperband  [ 1 ] 
Johnston  [57] 

Kanarick,  et  a/.  [59] 

McRae  [68] 

Williges,  et  at.  [94] 


Kanarick,  et  a/.  [59] 


1 The  classification  of  studies  throughout  this  document  represent  the  judgments  of  the  authors.  No 
formal  rating  system  was  used.  The  discussions  include  some  material  from  survey-type  and  other  references 
in  addition  to  the  studies  classified  in  the  tables. 
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One  common  conclusion  in  the  research  on  team  training,  regardless  of  whether  it 
was  based  on  an  “Organismic”  or  a “Stimulus-Response”  viewpoint,  is  that  individual 
proficiency  is  the  basis  for  an  effective  team  [59],  Thus,  in  emergent  situations,  devel- 
oping team  awareness  or  abilities  to  deal  with  unexpected  problems  requires  that  each  team 
member  has  attained  the  requisite  job  skills.  Team  training  is  likely  to  proceed  most  effi- 
ciently when  the  team  members  have  thoroughly  mastered  their  own  specific  assignments 
beforehand  [62],  Horrocks,  et  al.  [52,  53]  found  that  when  team  coordination  was 
emphasized  early  in  training,  it  interfered  with  acquisition  of  individual  task  competence. 

The  question  could  then  be  asked  whether  team  training  was  of  any  benefit  at  all 
and,  if  so,  to  what  degree  and  in  what  manner.  Is  it  possible  to  predict  when  individual 
training  will  be  superior  to  team  training?  [13]  Briggs  and  Johnston  [11]  concluded  their 
review  finding  no  direct  evidence  for  the  superiority  of  team  training  over  individual  train- 
ing. They  interpreted  this  finding  as  indicating  a need  for  team  training  devices  to  include 
provisions  for  either  refreshing  or  augmenting  individual  skills.  In  other  laboratory  investi- 
gations (derived  from  established  situations)  which  compared  individual  with  team  training, 
results  were  equivocal.  Horrocks,  et  al.  [52]  found  that  if  a member  of  an  intact  team 
was  replaced  by  another  equally  competent  person,  there  was  no  decrement  in  team  per- 
formance. This  implied  that  there  was  no  generalized  “team  skill”  that  operated  inde- 
pendently of  specific  task  competencies  [11].  Evidence  supporting  this  finding  came  from 
other  AIR  laboratory  studies  [63,  64],  in  which  only  the  proficiency  level  of  the  members 
at  the  start  of  team  training  determined  team  performance.  In  task  situations  that  were 
routine  and  well-established,  team  performance  was  seen  to  be  the  sum  total  of  individual 
performances  [49], 

On  the  other  hand,  Johnston  [57]  studied  two-person  teams  working  in  a simulated 
radar  situation  where  the  objective  was  to  track  and  intercept  incoming  aircraft.  The 
amount  of  coordination  and  communication  in  the  training  task  was  varied  and  demon- 
strated that  team  training  was  better  than  individual  training  when  the  criterion  task 
required  coordination  between  individuals.  In  such  emergent  situations,  training  in  “team 
skills”  (e.g.,  coordination)  was  necessary  and  effective  performance  was  regarded  as  some- 
thing more  than  the  sum  of  individual  skills  [68], 

Verbal  communication  training  was  found  not  to  be  useful  when  the  task  did  not 
require  the  skill  [57].  In  a study  by  Williges,  et  al.  [94]  it  was  concluded  that  verbal 
communication  played  an  insignificant  role  in  team  work  and  this  role  was  not  enhanced 
by  verbal  training.  In  this  study,  two-person  teams  were  required  to  coordinate  inter- 
ception of  two  planes  on  a simulated  radar  screen.  In  one  condition  team  members  received 
only  verbal  communication  concerning  the  performance  of  the  other  team  members;  in 
another  condition  the  team  members  were  given  verbal  communication  and  visual  access 
to  the  radar  screen  of  the  other  member.  It  was  found  that  verbal  communication  facili- 
tated performance  of  the  team  only  when  no  other  more  efficient  communication  channel 


was  available.  In  the  conditions  where  verbal  and  visual  communication  both  were  allowed, 
verbal  communication  had  no  effect  on  team  performance. 

Team  training  should  be  effective,  according  to  Alexander  and  Cooperband  [1],  when 
the  training  stresses  the  acquisition  of  coordinative  skills,  and  the  tasks  being  trained  are 
such  that  formal  rules  cannot  be  stated  and  procedures  must  be  developed  by  the  team  in 
accomplishing  the  task.  Kanarick,  et  al.  [59]  proposed  three  phases  for  team  training: 
Initially,  there  is  the  need  to  train  individuals  in  the  procedural 
aspects  of  their  jobs,  doctrine,  and  the  process  approach  to  decision 
making.  This  training  should  be  followed  by  a phase  in  which  team 
members  are  instructed  as  a unit,  learning  the  interactive  and  com- 
municative requirements  of  team  functioning.  The  final  phase  is 


devoted  to  tactical  training  where  teams  are  taught  to  apply  their 
procedural  and  interactive  skills  to  certain  situations  requiring  inno- 
vative and  creative  behaviors. 
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4.  TEAM  SKILLS 
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Studies  discussed  in  relation  to  team  skill  training  are  classified  in  Table  2.  As  can 
be  seen,  training  in  team  skills  is  important  for  application  to  emergent  situations  [89, 
88],  What  are  these  skills  and  how  should  they  be  trained? 


Table  2 

Team  Skills: 

Classification  of  Studies  Reviewed  in  Chapter  2,  Section  4 
TRAINING  FOCUS 


Team  Multi-Individual 


■■m 

Alexander  & Cooperband  [1] 
Boguslaw  & Porter  [101 
Federman  & Siegel  [331 
Kanarick,  et  at.  [591 
Sidorsky  & Houseman  [88] 
Sidorsky  & Simoneau  [891 

Hausser,  et  at.  [50] 

s 
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With  respect  to  one  such  skill— cooperation— Alexander  and  Cooperband  [ 1 1 have 
described  it  as  follows: 

Learning  to  cooperate  means  learning  the  strengths  and  weaknesses 
of  one  another,  learning  when  the  others  want  help  and  when  they  do 
not  want  it,  learning  to  pace  one’s  activities  to  fit  the  needs  of  all,  and 
learning  to  behave  so  that  one’s  actions  are  not  ambiguous. 

Nonetheless,  one  of  the  problems  *in  the  area  of  “team  skills”  is  lack  of  unambiguous, 
operational  definitions  of  skills  such  as  coordination,  interaction,  and  cooperation  [33]. 

As  there  are  conflicting  views  regarding  these  terms,  so  too  are  there  problems  in  train- 
ing and  evaluation  of  these  skills  [97], 

One  of  the  most  difficult  skills  to  train,  yet  one  of  the  most  important  in  dealing 
with  team  functions,  is  skill  in  the  analysis  of  one’s  own  errors.  Boguslaw  and  Porter  [10] 
reported  that  it  did  not  appear  possible  td  specify  how  to  train  this  skill.  It  was  hypo- 
thesized that  it  would  help  to  present  trainees  with  numerous  unusual  situations  which 
could  not  be  handled  by  simple  adherence  to  standard  operating  procedures.  Problem- 
solving discussions  (rather  than  didactic  presentations)  also  could  have  helped.  An  effec- 
tive team  performer  must  learn  when  he/she  is  reaching  the  point  of  being  overloaded  and 
needs  assistance.  Studies  are  needed  to  examine  the  kinds  of  “self-evaluative”  skills  which 
can  enhance  team  performance  and  to  identify  techniques  that  could  be  used  to  train 
these  skills. 

Training  in  team  awareness,  a skill  considered  important  to  performance  in  emergent 
situations  [59],  can  benefit  from  the  application  of  several  existing  technologies.  The 
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split-  or  multiple-screen  technique  of  television  can  be  used  to  show  team  members  how 
their  jobs  relate  to  other  team  members.  With  this  technique,  a trainee  could  observe 
how  a given  action  serves  as  a stimulus  for  some  other  team  member.  Such  training  gives 
trainees  a better  appreciation  of  their  individual  roles  in  the  team  and  of  the  need  for 
effective  communication  and  interaction. 

In  another  study,  the  PLATO  IV  Computer  Assisted  Instruction  (CAI)  system  was 
applied  to  training  interpersonal  skills  [50],  The  content  areas  chosen  included  feedback, 
communication,  goal  setting,  problem  solving,  decision  making,  and  reinforcement.  Train- 
ing had  a positive  effect  on  some  of  the  skill  performances  of  company  commanders  in 
leading  recruit  companies  through  a nine-week  training  period.  However,  in  most  of  the 
behavioral  measures  of  company  commander  performance,  CAI  training  did  not  have  a 
discernible  effect.  Further  investigation  of  the  usefulness  of  CAI  as  an  approach  to  train- 
ing team  skills  appears  needed. 


5.  SIMULATION  FIDELITY 

Studies  discussed  in  terms  of  simulation  fidelity  are  classified  as  shown  in  Table  3. 


Table  3 

Simulation  Fidelity: 

Classification  of  Studies  Reviewed  in  Chapter  2,  Section  5 


TRAINING  FOCUS 


Multi-Individual 


8riggs  & Johnston  (11,  12,  13] 
Briggs  & Naylor  (14] 


Alexander  & Cooperband  [1] 
Bachrach  (3] 


Madden  (69] 

Prophet  & Caro  182] 


In  training  teams  or  studying  their  behavior  in  emergent  situations,  high-fidelity  simu- 
lations may  be  contemplated  [69,  82],  High-fidelity  simulation  can  be  both  costly  and 
time  consuming;  space,  equipment,  and  staffing  require  substantial  investments  in  money 
and  time.  The  problems  of  measurement  are  great  because  of  the  interactions  of  the  many 
variables  involved.  Once  a large-scale  simulation  exercise  has  begun,  the  experimenters 
have  little  control  over  the  events  that  follow  1 3] . 

High-fidelity  simulation  of  complex  team  tasks  is  exemplified  by  those  simulating  air 
traffic  control  tasks  and  air  defense  operations  which  utilized  teams  ranging  from  30  to  40 
people.  These  tasks  very  closely  approached  the  complexities  of  the  operational  task  and 
task  fidelity  was  very  high,  since  in  some  cases  an  operational  system  itself  was  used  in 
training  [ 1 ] . 
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Can  low-fidelity  simulations  be  used  adequately  for  team  training?  A series  of  experi- 
ments by  Briggs,  et  al.  [11,  12,  13,  14]  led  to  the  conclusion  that  team  training  did  not 
transfer  well  to  the  operational  environment  when  the  training  simulation  was  of  low 
fidelity.  However,  all  these  studies  were  performed  in  contexts  derived  from  established 
situations.  No  studies  were  found  using  low-fidelity  simulations  in  emergent  situations. 
Recent  techniques  used  in  the  Army  to  simulate  two-sided  combat  engagements  have 
employed  low-cost,  high-fidelity  procedures  in  emergent  contexts.  These  techniques  will 
be  discussed  further  in  later  chapters  of  this  report. 


6.  FEEDBACK/KNOWLEDGE  OF  RESULTS 

Studies  on  performance  feedback  discussed  in  this  section  were  classified  as  shown 
in  Table  4.  Most  of  this  research  was  performed  in  laboratory  settings. 

Table  4 

Feedback/Knowledge  of  Results: 

Classification  of  Studies  Reviewed  in  Chapter  2,  Section  6 

TRAINING  FOCUS 

Team  Multi-Individual 


Burnstein  & Wolff  [15] 


Nebeker,  et  al.  [74] 


Cockrell  [191 
Cockrell  & Sadacca  [201 


Burnstein  & Wolff  [15] 
Wolff,  et  at.  (95,96) 
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Egerman,  et  al.  [31,  321 
Glanzer  [42] 

Glaser,  et  al.  [401 

Klaus  & Glaser  [62,  63,  64] 

Klaus,  et  al.  (65) 

Marra  [70] 

Short,  et  al.  [86] 

Wolff,  et  al.  [95,  96] 


Alexander  & Cooperband  [ 1 ] 
Kanarick,  et  al.  [59] 


t 


“Performance  feedback  is  unquestionably  the  single  most  critical  parameter  in  team 
or  individual  training,”  concluded  Kanarick,  et  al.  [59]  in  a review  of  the  research  litera- 
ture on  feedback.  The  series  of  studies  by  AIR  [31,  32,  40,  62,  63,  64,  86]  considered 
an  important  characteristic  of  teams  to  be  the  fact  that,  regardless  of  how  a correct  team 
response  was  made,  every  member  of  the  team  was  exposed  to  the  same  reinforcing 
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event— even  when  some  team  members  made  errors  or  were  idle.  These  studies  demon- 
strated that,  on  occasion,  team  reinforcement  might  foster  unwanted,  inappropriate 
responses  that  could  create  a significant  decrement  in  team  performance.  The  results  of 
this  program  of  research  suggested  several  guidelines  which  should  be  employed  for  team 
training: 

...  it  appears  essential  that  team  practice  result  in  clear  and 
immediate  reinforcement  following  each  correct  team  response;  prac- 
tice in  the  absence  of  team  reinforcement  for  criterion-level  perform- 
ance is  more  than  likely  to  lead  to  a decrement  in  team  proficiency  [62]. 

An  approach  used  to  provide  team  members  with  feedback  or  “knowledge  of  results” 
is  the  post-exercise  debriefing.  Debriefing  sessions  following  team  training  sessions  Eire  an 
opportunity  for  the  team  to  examine  individual  proficiencies  and  to  explore  alternative 
ways  of  organizing  the  task  so  as  to  develop  more  efficient  and  proficient  team  performance. 
However,  the  post-exercise  discussion  as  feedback  may  have  an  undesirable  characteristic— 
a large  time  gap  between  the  response  and  the  feedback  [42],  The  amount  of  elapsed  time 
before  feedback  is  given  should  be  studied  systematically  to  determine  the  effects  of 
delayed  feedback. 

The  results  in  a study  by  Nebeker,  et  al.  [74],  showed  that  although  individuals  per- 
formed better  with  feedback  than  without,  it  did  not  matter  whether  the  feedback  was 
raw  or  percentile  scores  or  whether  it  concerned  the  individual,  his/her  group,  or  both. 

The  results  underscored  the  importance  of  feedback  as  a general  factor  which  augmented 
and  sustained  performance.  Other  studies  of  feedback  have  demonstrated  that  reinforce- 
ment contingencies  affect  team  performance,  and  have  illustrated  the  effectiveness  of 
feedback  or  knowledge  of  results  to  sustain  performance  [15,  65,  70,  95,  96] . 

Team  members  receive  feedback  information  from  intrinsic  and/or  extrinsic  sources. 
Extrinsic  feedback  is  provided  by  a source  external  to  the  system,  such  as  an  instructor. 
Intrinsic  feedback  is  received  while  the  team  is  performing  its  tasks  and  is  inherent  in  the 
task  itself.  The  effectiveness  of  intrinsic  feedback  has  been  shown  in  a variety  of 
studies  [19,  20]. 

Providing  feedback  in  team  training  presents  problems  beyond  those  encountered  in 
training  individuals: 

These  problems  arise  from  three  considerations.  (1)  There  may 
be  several  criteria  of  effective  team  performance  with  no  clear-cut 
trade-offs  among  them;  these  criteria  may  be  vague  and  difficult  to 
state  objectively  and  may  change  during  system  operations.  (2)  In 
order  for  a team  to  operate  effectively,  it  is  necessary  for  its  members 
to  develop  and  maintain  individual  skills  as  well  as  skill  in  working 
together;  there  is  a possibility  that  these  skills  may  require  different 
feedback  procedures  which  may  mutually  interfere.  (3)  When  a 
complex  system  operates,  there  is  usually  a large  volume  of  informa- 
tion available  about  the  state  of  the  environment,  the  state  of  the 
system,  and  the  performance  of  system  personnel;  some  of  this  infor- 
mation may  be  conducive  and  some  inimical  to  effective  learning  [ 1 ] . 

A major  source  of  difficulty  in  team  training  is  the  identification  and  correction  of  indi- 
vidual errors.  In  many  complex  team  tasks,  there  are  often  many  possible  correct  pro- 
cedures, making  error  identification  difficult.  These  conditions  give  rise  to  the  problem 
of  monitoring  or  supervising  team  responses  to  evaluate  their  adequacy. 

Although  the  studies  cited  discuss  the  general  effectiveness  of  feedback,  more  work 
needs  to  be  done  to  systematically  investigate  the  methods  that  might  be  used  to  deliver 
feedback  information.  Various  techniques  such  as  videorecording  and  post-training  brief- 
ings are  now  used  to  inform  team  members  about  their  responses.  These  and  other 
methods  need  to  be  studied  regarding  their  effects  on  team  performance. 
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7.  TEAM  STRUCTURE  AND  COMPOSITION 


Studies  discussed  with  reference  to  team  structure  and  composition  were  classified 
as  shown  in  Table  5.  Although  not  directly  related  to  team  training  techniques,  questions 
regarding  the  effects  of  group  structure  and  composition  upon  training  are  important  for 
understanding  the  effects  of  training  upon  team  performance  [6], 

\ 

Table  5 

Team  Structure  and  Composition: 

Classification  of  Studies  Reviewed  in  Chapter  2,  Section  7 

TRAINING  FOCUS 


< 

=> 


co 

* 

C/5 

< 

I- 


Team 

Multi-Individual 

Bolin,  et  at.  [91 

Bolin,  et  at.  [9] 

Briggs  & Johnston  [11] 

Nebeker,  ef  at.  [74] 

Briggs  & Naylor  [14] 

Egerman,  et  at.  [31,32] 

Johnston  & Briggs  [58] 

Klaus  and  Glaser  [62] 

George  [38] 

Baldwin,  era'.  [6] 

Moore  [72] 

Two  types  of  team  structure  were  identified  and  studied  in  a series  of  laboratory 
experiments  [11].  In  a serial  or  vertical  structure,  activities  are  sequential  with  input  from 
one  team  member  based  on  output  of  another.  Parallel  team  structures  (e.g.,  rowing  team) 
do  not  have  the  same  member  interactions.  Briggs  and  Naylor  [14]  suggested  that  parallel 
team  structures  are  preferable  to  serial  structures,  because  team  performance  in  the  parallel 
structure  is  not  dependent  on  the  least  skilled  member.  However,  Klaus  and  Glaser  [62] 
reported  that  the  parallel  structure  led  to  only  a short-term  gain  and  eventually  to  a decre- 
ment in  team  performance. 

A variety  of  studies  on  team  structure,  workload  allocation,  physical  arrangement, 
and  member  replacement  yielded  conflicting  results  [47,  48].  For  example,  Bolin,  et 
al.  [9]  found  in  studying  task  size  and  team  performance  that  fully  cooperative  three-man 
teams  performed  better  than  individuals  working  alone  on  image  interpretation.  Moore 
[72]  and  George  [38]  studied  the  effects  of  task  characteristics  on  team  performance  in 
emergent  settings.  Moore  found  that  increasing  the  task  load  on  teams  did  not  affect 
team  performance  when  all  the  information  necessary  for  task  completion  was  furnished 
to  the  team.  George  showed  teams  performed  better  when  they  were  required  to  use  a 
communication  network  that  included  all  of  the  team  members.  In  a study  conducted  in 
an  established  task  context  to  test  the  effects  of  team  arrangement  on  performance  of 
radar  controller  teams  [58],  investigators  found  that  team  communication  inhibited  team 
performance  in  some  cases. 
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In  several  studies  occurring  in  established  task  situations,  the  effect  of  team  member 
replacement  was  found  to  be  relatively  unimportant  and  dependent  on  the  skill  levels  of 
the  persons  involved  [30,  31,  32,  48],  However,  personnel  turnover  was  considered  to  be 
a major  factor  degrading  team  performance  in  an  operational  setting.  For  example,  a tank 
commander  reported  that  with  40%  turnover  in  crews  every  90  days,  one  cannot  expect 
acceptable  levels  of  team  proficiency  [26].  There  is  a need  for  a systematic  study  of  the 
effects  on  team  proficiency  of  the  length  of  time  a team  performs  together  under  estab- 
lished or  emergent  conditions. 

With  such  conflicting  findings,  the  search  for  superior  team  training  methods  has  not 
made  much  progress.  Task-  or  situation-specific  characteristics  may  account  for  many  of 
the  divergent  findings  obtained  in  team  training  research.  A more  systematic  approach  to 
the  research  area  appears  necessary.  This  suggestion  is  amplified  in  the  following  section. 


8.  SYSTEMS  APPROACH  TO  TEAM  TRAINING 

The  most  common  framework  for  research  in  the  team  training  field  has  been  to 
study  team  behavior  in  the  same  manner  in  which  individual  behavior  is  investigated.  The 
“Stimulus-Response”  adherents  viewed  the  team  as  a “single  response  unit”  [62],  The 
team  product  as  a whole,  rather  than  its  component  parts,  was  the  focus  of  research.  This 
made  it  feasible  to  investigate  the  effects  of  various  conditions  of  feedback  upon  team  pro- 
ficiency. The  fact  that  the  team  response  might  have  been  quite  complex  and  might  have 
resulted  from  the  integration  of  separate  responses  of  several  team  members  was  of  interest 
only  to  investigators  who  followed  the  “Organismic”  viewpoint. 

Both  sets  of  researchers  favor  the  application  of  a “systems  approach”  or  Instruc- 
tional System  Development  (ISD)  model  to  the  design  and  development  of  team  training 
exercises,  materials,  methods,  and  devices.  Implementing  such  an  approach  could  yield 
information  needed  to  identify  critical  variables  to  be  studied  as  part  of  an  extensive  pro- 
gram of  team  training  research. 

Teams  are  created  or  defined  to  accomplish  certain  goals  or  missions  [36],  The 
relationships  among  team  members  are  meaningful  only  to  the  extent  that  they  are 
involved  in  attainment  of  the  goals.  Thus,  before  any  team  training  is  undertaken,  an 
in-depth  analysis  of  the  system  should  be  performed  [81].  The  task  characteristics  need 
to  be  identified  and  training  objectives  derived  [46,  58] . The  members  of  the  team 
should  be  made  aware  of  the  team’s  goals  and  objectives.  Exercises  and  instructional 
materials  should  be  structured  with  respect  to  the  objectives,  and  the  training  situation 
should  be  made  as  similar  as  possible  to  the  operational  criterion  situation  [8,  42].  Finally, 
the  quality  control  components  of  the  system  should  provide  feedback  data  to  trainees 
and  trainers  for  the  purpose  of  improving  the  training  system  and  team  proficiency. 

The  major  difficulties  to  overcome  in  order  to  implement  an  ISD  approach  to  team 
training  are  its  expense,  the  need  for  qualified  personnel,  and  the  problem  of  obtaining 
the  resources  and  authority  to  carry  out  the  training  research  and  development  deemed 
necessary. 


SUMMATION 


Two  approaches  to  the  study  of  team  training  have  been  described.  These  research 
trends  were  based  upon  what  was  termed  the  “Stimulus-Response  (S-R)”  and  “Organis- 
mic” models.  The  Stimulus-Response  model  led  to  laboratory  investigations  which 
addressed  team  training  problems  occurring  in  established  task  situations.  Adherents  of 
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the  Organismic  model  tended  to  study  team  training  in  more  realistic  surroundings,  as 
they  were  more  interested  in  the  emergent  phenomena  present  in  the  task  situation. 

Despite  their  conceptual  and  methodological  differences,  the  studies  performed  by 
both  groups  of  investigators  suggested  the  following  general  conclusions. 

• Team  training  is  a necessary  addition  to  individual  training  for  tasks  which 
require  interaction  and  other  “team  skills.” 

• Effective  team  training  can  occur  only  if  the  team  members  enter  the  train- 
ing situation  with  the  individual  skill  competencies  that  are  needed. 

• The  team  context  is  not  the  appropriate  location  for  initial  acquisition  of 
skills  by  individuals. 

• Performance  feedback  is  critical  to  team  as  well  as  individual  acquisition 
of  skills. 

The  research  findings  discussed  in  this  chapter,  indicating  that  a particular  team 
structure  is  “better”  than  others  or  that  one  type  of  reinforcement  schedule  is  more 
effective  than  another,  are  quite  fragile  results.  They  can  be  largely  obviated  by  such 
things  as  lack  of  fidelity  in  the  training  situation  [16].  The  nature  of  the  training  situa- 
tion is  a critical  element  for  future  team  training  research.  Up  to  now,  the  tasks  used  to 
study  team  training  have,  with  some  notable  exceptions,  been  quite  limited. 

Recent  programs  in  the  Army  have  demonstrated  the  possibility  of  developing  simu- 
lations that  are  high  in  fidelity,  yet  low  in  cost.  Such  simulations  may  provide  team  train- 
ing researchers  with  the  controls  they  need  to  study  “team  skills”  in  realistic  environments 
rather  than  in  the  laboratory  [26].  Other  implications  for  research  generated  by  the  liter- 
ature review  will  be  discussed  further  in  Chapter  5. 

The  review  of  the  research  literature  has  shown  that  weaknesses  exist  in  conceptual 
definitions  of  such  terms  as  “team,”  “coordination,”  “fidelity,”  and  “interaction.” 

Efforts  are  needed  to  provide  a more  precise  taxonomic  underpinning  in  this  field,  to 
serve  as  a framework  for  more  generalizable  and  comparable  research. 
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Chapter  3 


REVIEW 


OF  EVALUATION  TECHNIQUES 
FOR  TEAM  TRAINING 
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The  importance  of  methods  for  systematically  evaluating  team  training  effectiveness 
and  the  difficulty  in  developing  such  methods  have  been  widely  acknowledged  for  a number 
of  years  [26,  41,  42,  65,  76],  In  1955,  Glanzer  and  Glaser  [43 J noted  the  lack  of  satis- 
factory proficiency  measures  for  team  performance.  In  1962,  Glaser  [39]  remarked: 

With  respect  to  proficiency  measurement,  less  work  has  been 
done  in  this  area  than  has  been  carried  out  with  individual  per- 
formance, yet  the  importance  of  developing  proficiency  measures 
for  multiman  systems  is  being  more  frequently  considered. 

By  1974,  Obermayer,  et  at.  [76]  commented  that  an  economically  acceptable  means 
of  objectively  measuring  behavioral  skills  in  the  team  training  setting  was  an  elusive  goal. 

This  conclusion  was  echoed  by  the  Defense  Science  Board’s  1975  review  [26]  of  research 
and  development  programs  in  military  training.  Their  report  stated  that  the  team  per- 
formance measurement  problem  is  a “fundamental  stumbling  block  to  progress”  in 
improving  team  training. 

While  a systematic,  widely  applicable  methodology  for  team  training  evaluation  is  not 
yet  at  hand,  there  appears  to  be  a reasonable  consensus  as  to  what  the  methodology  ought 
to  accomplish  [8,  18,  60,  76].  The  literature  reviewed  suggested  that  any  team  perform- 
ance measurement  system  must  address  the  following  areas: 

• The  definition  of  team  performance  objectives  in  terms  of  specified,  obser- 

able  outcomes,  to  include  criteria  for  acceptance  and  conditions  of  performance. 

• The  definition  of  a metric  or  range  of  values  applicable  to  each  specified 
observable  event. 


• The  detection,  measurement,  and  recording  of  the  value  of  an  observed  event 
at  each  occurrence. 

• An  evaluation  of  the  team  as  having  attained  or  not  attained  the  objective- 
based  on  discrepancies  between  outcome  criteria  and  observed  event  values. 

• The  feedback  of  team  performance  data  to  the  training  environment. 

The  remainder  of  this  chapter  will  deal  with  current  developments  in  team  training 
evaluation  in  the  first  four  areas  listed.1  In  terms  of  the  two-dimensional  categori:ition 
of  training  focus  and  task  situation,  most  of  the  studies  reviewed  in  this  chapter  are 
related  to  the  evaluation  of  team  training  in  emergent  task  situations  (Table  6). 


DEFINITION  OF  TEAM  PERFORMANCE  OBJECTIVES 

Much  has  been  written  on  procedures  for  deriving  training  and  performance  objec- 
tives [2,  91] . The  actual  statements  of  objectives  are  the  end  products  of  a series  of  steps 
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The  feedback  of  team  performance  data  was  discussed  in  Chapter  2. 
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Table  6 


\ 


Team  Training 

Classification  of  Studies/Procedures  Reviewed  in  Chapter  3. 
TEAM  TRAINING 

Team  Multi-lndividur: 


Project  IDOC  (tank  gunnery) 

ARTEP  Validation  Studies  [51] 

CARMONETTE  (computer  simulator 
models)  [21] 

Combat-Ready  Crew  Performance 
[Obermayer's  study,  76] 

MASSTER 

METTEST  (investigation  of  types  of 
aggregate  models) 

NORM  Project  [84] 

REALTRAIN  (crew-served  weapons) 
[29,  87] 

SAGE  studies  described  by  Biel  [8] 
TACDEW  System  [18,  34,  35] 

UPAM  (casualty  assessment  method) 

MILES  [29] 

REALTRAIN  (rifle  studies)  [29] 

in  a systems  approach  to  training  development.  Under  this  approach,  the  resulting  objec- 
tives have  three  characteristics: 

• They  must  describe  behaviors  that  are  to  be  demonstrated  in  the  test  situation. 

• They  should  indicate  the  conditions  under  which  these  behaviors  are  to 
be  demonstrated. 

• They  should  include  criteria  or  standards  of  performance. 

The  absence  of  performance  criteria,  usually  stated  in  terms  of  accuracy,  time,  casualties 
taken,  tolerance  limits,  and  so  forth,  appears  to  be  a seriously  deficient  aspect  of  the 
evaluation  of  team  training  and  performance. 

The  Army  Training  and  Evaluation  Program  (ARTEP)1  , a body  of  training  reference 
literature  stressing  a mission-oriented,  performance-based  approach,  has  been  under  devel- 
opment since  1974.  In  1974-75,  the  9th  Infantry  Division  and  the  1st  Cavalry  Division 
trained  and  tested  under  prototype  ARTEPs  as  part  of  an  extensive  examination  of  the 
feasibility  of  ARTEP  implementation.  One  of  the  primary  objectives  of  the  effort  was  to 
“assess  the  effectiveness  of  the  ARTEP  as  an  instrument  for  evaluating  unit  proficiency”  [51]. 


1 Sec  Chapter  4 for  a more  detailed  description  of  ARTEP. 
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It  was  concluded  that  the  standards  of  performance  stated  for  ARTEP  missions  needed 
revision.  The  standards  were  inaccurate  in  some  instances  and  too  general  and  vague  in 
many  others,  requiring  individual  rater  judgment  to  a greater  extent  than  necessary.  Army 
Training  and  Evaluation  Program  standards,  while  ostensibly  stated  in  specific  perform- 
ance terms,  contained  many  indefinite  terms  such  as  “on  time,”  “excessive,”  “sufficient,” 
“promptly,”  and  “proper.”  Without  some  guidance,  interpretation  of  such  terms  is  cer- 
tain to  vary  widely.  One  major  difficulty  relates  to  the  use  of  multiple  standards  and  the 
lack  of  guidance  as  to  the  relative  value  of  each  component.  Such  heavy  reliance  upon 
judgments  of  individual  evaluators  can  lead  to  diminished  Army  Training  and  Evaluation 
Program  validity  and  reliability. 

Obermayer  et  al.  [76],  describing  a highly  sophisticated,  semi-automated  system  for 
evaluating  air  crew  performance,  stated  that  in  some  areas,  such  as  air  combat  maneuvering, 
combat  readiness  determination  is  purely  subjective.  Chesler  [18]  asserted  that  standards 
for  many  combat  situations — simulated  or  real — are  often  matters  of  widely  differing 
opinion.  Larson  [67]  described  a possible  way  to  overcome  these  differences.  In  his  sur- 
vey of  team  performance  effectiveness  measures,  he  suggested  the  use  of  the  Delphi  tech- 
nique—a method  for  systematically  extracting  expert  opinion— to  elicit  subjective  indicators 
of  team  performance  from  senior  Marine  Corps  personnel.  These  performance  indicators 
could  then  be  used  in  field  exercises  by  less  experienced  evaluators. 


DEFINITION  OF  OBSERVABLE  EVENT  METRICS 

The  critical  concern  in  proficiency  evaluation  is  whether  a team  has  or  has  not  gotten 
the  job  done.  It  is  the  objective  of  evaluation  indices  to  measure  data  that  indicate 
whether  the  defined  mission  has  been  successfully  performed.  Precise  specifications  of 
events  to  be  assessed  could  be  advanced  by  implementing  the  Instructional  System  Devel- 
opment (ISD)  approach,  which  subsumes  the  use  of  job/task  analysis  procedures.  However, 
much  performance  evaluation  in  training  situations  today  is  still  highly  subjective  [18]. 

Upon  examination,  this  problem  inevitably  boils  down  to  a lack  of  criterion  perform- 
ance variables  that  are  objective,  recordable,  discriminatory,  and— most  important— accept- 
able to  a consensus  of  persons  familiar  with  the  tasks  and  skills  of  concern.  This  is  the 
criterion  dilemma,  which  is  especially  severe  for  team  training.  Acceptable  indices  of  good 
and  poor  performance  for  military  operations  are  difficult  to  define  [18],  Nonetheless, 
some  techniques  for  the  systematic  identification  of  measurable,  related  events  have  been 
developed  [8,  18,  76], 

Adoption  of  such  methods  may  well  provide  the  kind  of  comprehensive,  detailed 
information  which  is  necessary  to  the  development  of  valid  performance  criteria,  but 
which  at  present  is  generally  not  available  to  training  and  evaluation  personnel.  The  Navy’s 
Tactical  Advanced  Combat  Direction  and  Electronic  Warfare  (TACDEW)  System  appears 
well  thought  out  and  could  readily  serve  as  a model  approach  for  other  team  performance 
evaluation  efforts. 

The  basic  events  in  the  TACDEW  evaluation  approach  [18]  are  the  “performance 
variable”  and  the  “situational  variable”: 

A performance  variable  is  a measure  of  the  accuracy  of  timeliness 
of  a trainee  response.  A situational  variable  is  a measure  of  some  aspect 
of  the  embedding  environment  for  the  trainee’s  response.  Performance 
variables  derive  from  two  types  of  data  elements:  actual  trainee  response, 
and  the  response  considered  correct.  The  performance  variable  is  the 
quantitative  difference  between  or  qualitative  comparison  of  the  two. 

For  example,  reported  bearing  (trainee  response  data  element)  for  a 
target  is  320  degrees;  the  true  target  bearing  (correct  respone  data 


element)  is  300  degrees;  the  performance  variable,  error  in  reported 
bearing,  is  20  degrees.  This  performance  variable  is  a measure  of 
accuracy  of  response. 

Performance  variables  relating  to  timeliness  of  trainee  response  are  derived  from  the 
actual  latency  of  the  trainee’s  response  and  a reference  or  ideal  response  latency.  A set 
of  performance  (and  situational)  variables  is  then  related  to  an  event.  Once  the  event 
occurs,  preliminary  lists  of  performance  variables  are  generated  to  identify  any  and  all 
variables  that  show  promise  of  serving  as  measures  of  the  attainment  of  training  objective. 
The  developers  of  the  TACDEW  system  anticipate  that  their  approach  will  help  clarify 
the  relationship  among  the  various  measurements  and  their  relevance  as  criteria  for 
making  adequate  judgments  regarding  team  training  outcomes. 

Using  similar,  but  perhaps  less  clearly  articulated  methods,  training  evaluators  in 
other  Services  are  attempting  to  define  objective-related  events.  For  example,  in  Project 
REALTRAIN,1  an  Army-supported  approach  which  uses  realistic,  two-sided,  free-play 
exercises,  a casualty  assessment  technique  was  developed  for  the  basic  infantry  weapon, 
the  M16A1  rifle.  The  development  of  the  casualty  assessment  technique  for  the  M16A1 
and  of  similar  techniques  for  other  infantry  and  armor  weapons  was  considered  a break- 
through necessary  in  simulating  the  tactical  environment  [29,  87].  As  part  of  the  REAL- 
TRAIN  procedures,  a sequential  record  of  events  is  kept  at  the  controller’s  station 
during  each  engagement.  This  record  includes  each  casualty,  the  time  at  which  it  occurred, 
and  the  weapon  used. 

Statistical  analysis  can  help  to  reduce  the  number  of  possible  variables.  Some  vari- 
ables may  correlate  highly  with  others,  and  can  be  eliminated.  Other  variables  may  show 
very  little  spread  in  range  of  scores,  so  they  do  not  differentiate  among  trainees.  Project 
NORM  (Normative  Operations  Reporting  Method)  was  an  effort  by  the  System  Develop- 
ment Corporation  [84]  to  identify  valid  performance  variables  for  SAGE  (Simulated  Air- 
Ground  Environment,  18)  team  evaluation,  and  to  determine  how  situational  variables 
affected  team  performance.  A basic  concept  of  Project  NORM  was  that  situational  vari- 
ables were  measures  of  exercise  difficulty.  An  exploratory  pilot  study  provided  data  for 
41  variables— 24  performance  and  17  situational  variables.  Correlational,  linear  multi- 
variate regression,  and  factor  analysis  techniques  were  used  to  reduce  the  original  number 
of  41  variables  to  13  [24],  Six  situational  variables  (related  to  the  task  at  hand)  were 
potential  contributors  to  performance,  and  seven  performance  variables  (e.g.,  detection 
latency)  were  considered  relevant  for  evaluating  system  performance.  While  the  specific 
variables  selected  appear  to  be  useful  primarily  to  the  study  content,  the  analytic  meth- 
odology employed  to  derive  these  variables  is  applicable  to  emergent  task  situations 
in  general. 

Biel  [8]  described  another  SAGE  study  in  which  all  measures  used  to  evaluate  team 
performance  in  a simulated  environment  were  collected  during  a series  of  specially  pre- 
pared simulated  exercises.  Correlations  among  team  performance  variables  with  measures 
of  overall  system  performance  in  detection,  identification,  tracking,  interceptor  commit- 
ment, and  interceptor  guidance  were  computed  and  the  resulting  matrix  factored.  The 
deletion  of  those  measures  which  did  not  have  a significant  relationship  to  overall  system 
performance  was  recommended.  It  was  recommended  that  a small  number  of  systemati- 
cally obtained  observations  be  used  to  make  judgments  about  team  performance. 

In  Army  tank  crew  training,  a large  number  of  performance  objectives  have  been 
developed  for  tank  gunnery  training  and  proficiency  assessment.  Project  IDOC  (Identi- 
fication of  Test  Doctrine  for  Cost-Effective  Qualification  of  Tank  Crews)  is  concerned 


1 See  Chapter  4 for  a further  discussion  of  REALTRAIN. 
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with  identifying  a reduced  set  of  tasks  on  which  to  base  the  development  of  cost-effective 
performance  measures  that  are  applicable  to  the  entire  set  of  objectives.  A joint  HumRRO/ 
American  Institutes  for  Research  effort  was  undertaken  to  determine  the  component  team 
behaviors  for  each  objective,  and  the  commonality  of  behaviors  across  objectives.’  Plans 
are  currently  being  formulated  to  analyze  data  obtained  so  far  on  generalities  across  tank 
gunnery  tasks,  psychometric  problems,  and  practical  constraints  on  performance  assessment. 


DETECTION,  MEASUREMENT,  AND  RECORDING  OF  OBSERVABLE  EVENTS 

A problem  of  some  concern  is  the  unreliability  and  inaccuracy  of  trainers  and 
observers  in  evaluating  team  performance  in  the  field.  Since  there  is  no  fixed  doctrine 
of  what  to  observe  and  how  to  evaluate  it  for  various  team  activities,  the  tendency  is  for 
inter-rater  reliability  to  be  quite  low. 

Larson’s  1971  review  [67]  of  the  team  performance  measurement  literature  concurs 
with  this  assessment.  He  stated  that  there  was  a shortage  of  objective,  quantitative  methods 
available  for  application  in  field  exercises,  caused  by  an  inherent  difficulty  in  measuring 
complex  human  performance  in  a field  environment.  In  REALTRAIN  field  exercises, 
detection  and  measurement  of  observable  events  (casualty  assessment)  are  performed 
entirely  by  human  controllers  [87].  The  light  anti-tank  weapon  (LAW)  and  90mm  recoiless 
rifle  (RCLR)  are  equipped  with  range-calibrated  sighting  plates  which  a controller  looks 
through  during  a simulated  engagement  to  determine  hits.  Casualties  with  the  anti-tank 
weapon  (TOW)  are  determined  by  a controller  sighting  through  an  accessory  telescope 
mounted  on  the  outside  of  the  weapon.  In  a recent  field  tryout  of  the  REALTRAIN 
method  for  combined  arms  tactical  training,  it  was  reported  that  the  data  collected  were 
limited  because  of  the  methodological  problems  of  quantitatively  measuring  team  per- 
formance [87]. 

Some  team  training  evaluation  efforts  attempted  to  circumvent  the  “unreliable” 
observer  problem  by  automating  detection  and  measurement  as  much  as  possible.  This  is 
particularly  evident  in  areas  where  sophisticated  electronic  and  computing  technology  are 
employed  in  mission  fulfillment  [4,  8,  18,  76].  The  Army  is  presently  investigating  the 
use  of  laser  technology  for  this  purpose.  Project  MILES  (Multiple  Integrated  Laser  Engage- 
ment System)  involves  the  integration  of  lasers  with  a variety  of  weapons  types  [29], 
“Firing”  of  a weapon  emits  a weak  laser  beam  which,  if  aim  is  accurate,  activates  a record- 
ing device  on  the  target.  The  MILES  system  is  still  experimental  and  has  not  been  sub- 
jected to  field  trials,  but  it  has  good  potential  for  solving  the  problem  of  unreliable  human 
measurement.  Even  though  MILES  was  developed  for  use  in  multi-individual  training 
settings,  the  concept  could  be  applicable  to  the  evaluation  of  crew-served  weapons  sys- 
tems training. 

Where  such  technological  solutions  are  not  to  be  used,  other  techniques  for  minimiz- 
ing rater  effect  are  being  explored.  To  support  the  development  of  systematic  methods 
of  criterion-referenced  performance  assessment,  the  Army  Research  Institute  through 
Project  METTEST  (Methodological  Issues  in  Criterion  Referenced  Testing)  is  funding 
several  investigations  into  the  applicability  of  psychometric  methods  to  the  requirements 
of  measuring  combat  team  performance.  One  such  effort  addresses  the  question  of  which 
tasks,  given  limited  time  and  resources,  should  be  assessed  via  high-fidelity  operational 
simulations,  which  may  be  measured  “synthetically”  (via  paper-and-pencil  instruments), 
and  which  may  be  assessed  by  inference  from  performance  on  related,  more  inclusive  tasks. 


1 Interview  with  Army  Research  Institute  staff. 
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EVALUATION  OF  OBJECTIVE  ATTAINMENT 

Attempts  to  evaluate  team  performance  during  field  performance  tests  have  varied 
considerably.  In  some  cases  it  is  assumed  that  if  a given  team  completes  an  exercise  suc- 
cessfully, the  team  had  received  training  in  relevant  behaviors. 

The  most  common  means  of  evaluating  performance  is  overall  subjective  evaluation. 
Where  detailed  checklists  are  used,  each  individual  team  member  receives  a numerically 
weighted  score.  The  team  score  then  is  calculated  through  some  weighted  combination  of 
individual  scores— generally  a simple  sum. 

The  ability  of  human  evaluators  to  effectively  judge  the  success  of  teams  in  complex 
field  exercises  is  generally  considered  unsatisfactory.  In  the  Army  Training  and  Evalua- 
tion Program  (ARTEP)  validation  report  [51]  cited  earlier,  it  was  concluded  that  the 
ARTEP  in  its  present  form  is  not  a standardized  test  instrument.  There  is  no  reason  to 
expect  that  different  teams  would  be  evaluated  under  the  same  conditions  when  using 
Army  Training  and  Evaluation  Program  guidance  and  standards.  The  standards  are  too 
subjective  and  evaluator  performance  is  too  erratic.  There  is  no  explanation  of  how  to 
relate  task  performance  to  overall  mission  performance  or  how  to  adjust  standards  to 
account  for  varying  test  conditions  [51]. 

The  Modem  Army  Selected  System  Test  Evaluation  and  Review  (MASSTER)  attempts 
to  evaluate  weapons  systems  training  by  employing  a combination  of  checklists,  associ- 
ated logic  trees,  and  expert  judges  in  the  field  environment.  The  checklist  logic  tree  eval- 
uation uses  field  observers  to  check  off  the  team's  performance  in  specified  situations. 

The  team  passes  or  fails  progressively  larger  tests  as  a result  of  their  performance  on  pre- 
vious examinations.  MASSTER  also  assigns  officers  to  subjectively  evaluate  large  teams 
during  an  exercise  [67], 

The  difficulties  currently  experienced  in  making  reliable  and  accurate  judgments  about 
team  performance  are  in  large  measure  due  to  the  multiplicity  of  potentially  influential 
factors.  Chesler  [18]  stated: 

. . . single  performance  measures  are  inadequate  for  making  overall 
evaluations  of  system  effectiveness,  presumably  because  multiple  factors 
are  involved  in  the  determination  of  mission  success  or  failure.  Com- 
bining measures  into  overall  indices  has,  so  far,  not  seemed  to  help  much, 
probably  because  the  relationship  between  them  is  not  clearly  under- 
stood. Thus,  lumping  them  together  does  not  necessarily  improve  the 
quality  of  system  evaluation. 

Recognition  of  this  complexity  has  led  to  a number  of  investigations  of  analytic  or 
modeling  approaches  to  the  evaluation  problem.  A study  within  the  Army  Research  Insti- 
tute’s METTEST  (Methodological  Issues  in  Criterion  Referenced  Testing)  project  is  con- 
cerned with  the  applicability  of  traditional  measurement  models  to  the  need  for  team 
performance  indices.  Measures  are  needed  that  give  valid  representation  to  the  many  team- 
related  and  environmental  factors  that  influence  mission  attainment.  Preliminary  work 
has  suggested  that  the  predominantly  linear  additive  models  of  psychometrics  are  not 
applicable  in  this  context— that  non-linear  (e.g.,  exponential)  and  “multiple-cutoff’  models 
may  be  more  appropriate.  In  “multiple-cutoff  models,”  team  activities  are  structured  in 
a manner  related  to  the  sequence  in  which  they  would  occur  in  the  field,  and  each  is 
evaluated  on  a GO/NO-GO  basis.  Within  this  approach  “compensatory”  and  “non- 
compensatory” activities  can  be  identified.  An  activity  is  non-compensatory  if  one  partic- 
ular component  will  abort  the  attainment  of  the  mission  if  it  is  not  performed  properly. 

If  some  other  component  can  successfully  pick  up  the  slack  left  by  the  failed  component, 
then  it  is  compensatory. 

The  UPAM  model  (Unit  Performance  Assessment  Model)  developed  at  the  Army 
Research  Institute  employs  casualty  data  as  a measure  of  both  the  achievements  of  a 
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team  (casualties  inflicted  on  an  opposing  force)  and  the  cost  to  the  team  (casualties  sus- 
tained) during  its  efforts  to  fulfill  a mission.  A team’s  accomplishments  cure  measured  by 
inputting  to  the  model  data  from  REALTRAIN  field  exercises  and  deriving  a single  value 
reflecting  both  cost  and  achievement.  A criterion  value  for  assessing  tne  empirical  result 
is  obtained  by  having  the  unit  commander  provide  estimates  of  these  values  that,  in  his 
judgment,  are  indicative  of  a successfully  accomplished  mission.  The  difference  between 
empirical  output  and  the  commander’s  “expected”  output  (on  a scale  from  -1  to  +1)  is 
viewed  as  an  index  of  the  team’s  success  in  attaining  its  goal.  The  UPAM  model  has 
been  applied  to  both  infantry  and  armor  units.  However,  in  its  present  state  of  develop- 
ment, the  model  does  not  provide  evaluative  information  concerning  team  activities  that 
contribute  to  or  detract  from  mission  accomplishment. 

The  most  detailed  and  sophisticated  analytic  model  found  in  this  survey  is  the 
CARMONETTE  ground  close  combat  computer  simulation  [21].  Its  parameters  include 
detailed  descriptions  of  team  size,  team  type,  firepower,  vulnerability,  mobility,  and  sensing 
capabilities.  Data  on  terrain  range  from  elevation  and  height  of  vegetation  to  trafficability. 
A set  of  “orders”  (i.e.,  a mission)  can  be  transmitted  to  each  team  of  opposing  forces. 

Each  team  can  acquire  information  and  relay  it  to  equivalent  teams  or  to  its  command 
headquarters.  While  the  output  from  CARMONETTE  provides  only  descriptive  data,  it 
could,  if  provided  with  field  data,  provide  valuable  clues  concerning  the  effect  of  per- 
formance and  situational  variables  on  mission  outcomes. 


SUMMATION 


There  is  still  a major  unsolved  problem  in  the  development  of  evaluation  methods  and 
tools  for  team  training— namely,  there  is  a need  for  reliable,  objective-based,  team  perform- 
ance measures.  The  literature  reviewed  indicated  the  difficulties  of  doing  research  in  this 
area  in  spite  of  its  acknowledged  importance  to  the  improvement  of  team  training.  The 
current  support  within  the  Army  Research  Institute  for  R&D  in  this  area  demonstrates 
their  realization  of  its  criticality.  Most  of  this  research  is  still  in  progress.  Findings 
gleaned  from  the  studies  surveyed  suggested  the  following: 

• A systems  approach  to  team  training  development  should  alleviate  some  of 
the  problems  associated  with  the  establishment  of  criteria/standards. 

• Much  performance  evaluation  in  team  training  today  is  highly  subjective 
and  unreliable. 

• Lacking  an  adequate  team  performance  measurement  system,  alternative  team 
training  programs  and  team  proficiency  cannot  be  properly  evaluated. 

• Automation  of  the  monitoring  and/or  data  recording  process  is  needed  in 
order  to  solve  the  problem  of  unreliable  human  observers. 

Recent  innovations  in  team  training  methodology  discussed  in  Chapter  2 promise 
significant  improvements  in  operational  performance.  A strong  emphasis  on  R&D  in 
team  performance  measurement  is  necessary  if  this  improvement  is  to  be  detected  and 
operationally  realized. 
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Examples  of  team  training  techniques  currently  in  use  in  the  Services  are  presented 
in  this  chapter.  It  is  not  intended  to  be  an  exhaustive  presentation  of  such  training 
methods.  Rather,  the  examples  were  chosen  to  represent  the  variety  of  methods  now 
employed  to  conduct  team  training.  The  training  devices  and  systems  discussed  are  cate- 
gorized in  Table  7.  All  were  classified  as  pertaining  to  emergent  task  situations  because 
they  stress  the  realistic  requirements  of  operational  environments. 


ARMY  TEAM  TRAINING 

Army  team  training  is  predominantly  a unit  training  function  with  no  formal  insti- 
tutional training  requirements.  The  great  majority  of  personnel  who  complete  Military 
Occupational  Specialty  (MOS)  training  at  Army  schools  and  training  centers  are  assigned 
to  units  as  replacements.  They  receive  team  training  in  the  units  in  the  form  of  individual 
on-the-job  training,  augmentation,  cross-training,  and  collective  (team)  training.  Most 
training  is  performed  on  operational  equipment  with  the  objectives  of  maximizing  pro- 
ficiency of  readiness  [27,  49], 

In  the  Army,  Advanced  Flight  Training  is  considered  team  training.  Graduates  of 
Undergraduate  Pilot  Training  receive  supplementary  training  in  the  specific  aircraft  they 
will  fly  on  operational  missions.  The  supplementary  training  is  provided  by  the  opera- 
tional unit.  The  new  pilot  is  integrated  into  the  operational  unit  where  he  undergoes 
transitional  training  as  part  of  normal  unit  training  [26], 

Much  of  the  team  training  conducted  in  the  Army  is  based  on  information  contained 
in  Army  Training  and  Evaluation  Program  (ARTEP)  publications.  That  is,  ARTEPs 
“drive”  the  Army’s  team  training  system.  ARTEPs  are  developed  to  assist  training  mana- 
gers and  trainers  in  the  preparation,  conduct,  and  evaluation  of  team  training.1  They  con- 
sist of  a series  of  training  and  evaluation  outlines  which  specify  for  a particular  element  of 
a battalion/task  force,  or  separate  company,  the  following  information: 

• The  unit  (i.e.,  crew,  squad,  platoon,  company/team  or  battalion/task  force) 
for  which  the  outline  is  applicable. 

• The  mission  to  be  performed. 

• The  general  conditions  (situation)  under  which  the  mission  will  be  performed. 

• The  primary  training  and  evaluation  standards  upon  which  the  unit  will  be 
evaluated  as  either  satisfactory  or  unsatisfactory  (Go/No  Go). 

• The  collective  training  objectives  applicable  in  the  performance  of  the  mis- 
sion; guidance  for  estimating  support  requirements  necessary  to  conduct 
formal  evaluations,  and  tips  to  trainers  and  evaluators.  [28] 


1 See  Chapter  3 for  a discussion  of  ARTEP  as  an  evaluation  mechanism. 
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TASK  SITUATION 


Table  7 


Classification  of  Training  Contexts 

TRAINING  SITUATION 


Team 


Multi-Individual 


i 

*2 


E 

uj 


CATTS  (Combined  Arms  Tactical 
Training  Simulator)  [26] 

C-5  Mission  Flight  Simulator 
C-130  Flight  Simulator 
C-141  Flight  Simulator 

FIST  (Functional  Integrated 
Systems  Trainer)  [26] 

Device  14A2,  Surface  Ship  Anti- 
submarine Warfare  (ASW)  Early 
Attack  Weapons  Systems  Team 
Trainer  [49] 

Device  14A6,  ASW  Coordinated 
Tactics  Trainer  [49] 

Device  21A37/4,  Submarine  Fleet 
Ballistic  Missile  (FBM)  Training 
Facility  [49] 

TACDEW  (Tactical  Advanced  Combat 
Direction  and  Electronic  Warfare 
System)  [34,  49] 

TESE  (Tactical  Exercise  Simulator 
and  Evaluator)  [83] 

WST  (B-52  Weapons  Systems  Trainer) 

WST  (Device  2F87,  Weapons  System 
Trainer)  [49] 

MILES  (Multiple  Integrated  Laser 
Engagement  System)  [29] 

REALTRAIN  [29] 

SCOPES  (Squad  Combat  Operations 
Exercise,  Simulation)  [28,  92] 

TWAES  (Tactical  Warfare  Analysis 
and  Evaluation  System)  [26] 

Inasmuch  as  the  Army  Training  and  Evaluation  Programs  (ARTEPs)  outline  the  basic 
missions  that  teams  should  be  able  to  perform  to  be  combat  ready,  they  are  a valuable 
source  of  information  in  preparing  the  scenarios  for  training  strategies/exerciscs  such  as 
SCOPES  (Squad  Combat  Operations  Exercise,  Simulation),  REALTRAIN,  MILES  (Multi- 
ple Integrated  Laser  Engagement  System),  and  CATTS  (Combined  Arms  Tactical  Training 
Simulator),  which  are  discussed  in  this  section. 
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1.  SCOPES  (Squad  Combat  Operations  Exercise,  Simulation) 

SCOPES  was  originally  designed  to  teach  movement  techniques  to  rifle  teams  and 
squads.  It  has  been  successfully  used  to  train  and/or  evaluate  soldiers  or  small  teams  on 
a wide  range  of  battlefield  skills,  tasks,  or  missions. 

Briefly,  SCOPES  [28,  92]  works  in  the  following  way.  Trainers  first  develop  a tacti- 
cal scenario  based  on  specified  training  objectives.  Six-power  telescopes  mounted  on  each 
rifleman’s  M16A1  rifle  are  used  to  identify  numbers  worn  on  the  helmets  of  all  partici- 
pants in  opposing  forces.  When  a number  of  an  “enemy”  soldier  is  identified,  the  soldier 
making  the  identification  fires  a blank  round  and  calls  out  the  number  he  sees.  Con- 
trollers in  the  vicinity  in  turn  pass  the  number  over  a radio  net  to  controllers  operating 
with  the  opposing  force.  If  the  number  is  correct,  the  controller  immediately  “kills”  the 
man  wearing  the  number  seen. 

With  this  technique,  “battles”  may  be  conducted  with  real  outcomes  that  depend 
on  the  tactics  selected  and  the  individual  techniques  used.  It  is  possible  to  objectively 
determine  “winners”  and  “losers.” 

After  each  engagement,  troops  on  both  sides  are  brought  together  for  an  after-action 
review  to  discuss  errors  made  and  lessons  learned.  In  this  manner,  the  correct  techniques 
to  avoid  being  “killed”  can  be  brought  out,  learned,  and  practiced  in  subsequent  engagements. 

2.  REALTRAIN 

The  REALTRAIN  program  permits  the  conduct  of  exercises  using  opposing  forces 
on  a larger  scale  than  does  SCOPES.  To  simulate  the  effects  of  other  weapons  systems, 
optical  devices  including  telescopes  and  plastic  sighting  plates  are  mounted  or  used  with 
light  anti-tank  weapons  (LAWs),  106mm  recoiless  rifles  (RCLRs),  anti-tank  weapons  (TOWs), 
tank  man  guns,  DRAGON  medium  anti-tank  weapons,  and  90mm  RCLRs.  These  devices 
are  aligned  with  the  weapon’s  sight  picture,  thereby  allowing  controllers  to  see  the  same 
sight  picture  as  the  gunners.  This  permits  them  to  verify  a gunner’s  aim  during  a target 
engagement.  Gunners  “shoot”  at  targets  by  announcing  the  identification  numbers  worn 
by  the  soldiers  aligned  in  their  sights.  Each  optical  device  is  correlated  with  the  maxi- 
mum effective  range  of  the  particular  weapons  system  with  which  it  is  used  [29]. 1 

Field  tests  conducted  using  SCOPES  and  REALTRAIN  exercises  have  proven  that 
engagement  simulation  techniques  work  for  training  multi-individual  groups.  Both  indi- 
vidual soldiers  and  units  reach  higher  levels  of  tactical  proficiency  more  rapidly,  and 
soldiers  are  motivated  to  train  because  the  tactical  exercises  present  a real  challenge. 

3.  MILES  (Multiple  Integrated  Laser  Engagement  System) 

MILES  is  a group  of  experimental  training  devices  that  use  an  eye-safe  laser  beam  to 
simulate  the  effects  of  weapons  expected,  on  the  modern  battlefield.  The  MILES  system 
will  permit  conduct  of  day  and  night  tactical  exercises  involving  battalion  and  task  force 
size  teams  [29],  It  is  scheduled  for  testing  in  1978. 

The  system  will  improve  the  integration  of  gunnery  techniques  into  tactical  training. 
Some  of  the  subsystems  are: 

• Vehicle  Engagement  System  (VES).  The  VES  will  be  used  by  armor  units 
and  anti-armor  weapons  such  as  TOW  (tube-launched  optically  tracked  wire 


1 A more  complete  description  of  REALTRAIN  can  be  found  in  Army  Training  Circular  71-5.  [29) 
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guided  missile),  DRAGON  (medium  anti-tank  weapon),  and  the  106mm 
recoiless  rifle. 

• Target  Engagement  Simulator  (TES).  The  TES  will  be  used  by  infantry 
troops  to  provide  engagement  realism.  It  consists  of  a laser  transmitter 
and  a hit  indication  device  which  signals  “kills”  and  “near  misses.” 

• Machine  Gun  Laser  (MGL).  The  MGL  simulates  machine  guns  both  unmounted 
and  mounted  on  tanks. 

The  MILES  procedure  (as  well  as  SCOPES  and  REALTRAIN)  focuses  on  individual 
skills  training  in  emergent  task  situations.  The  feasibility  of  using  these  techniques  for 
training  team  skills  needs  to  be  determined. 

4.  CATTS  (Combined  Arms  Tactical  Training  Simulator) 

The  purpose  of  the  CATTS  system,  which  is  still  under  development,  is  to  provide 
battalion  commanders  and  their  staffs  with  simulated  combat  situations  operating  from 
a ground  command  post. 

The  CATTS  system  has  several  features  desired  for  training  [26],  It  allows  realistic, 
real-time  ground  combat  simulation;  realistic  mockups  of  command  post  vehicles;  actual 
field  radios  for  communication;  on-the-spot  command  decisions  and  critiques;  extensive 
automation  to  assist  controllers;  and  cost-effective  software  and  hardware. 

The  simulator  utilizes  a Xerox  Sigma  9 computer  to  meet  the  following  training 
objectives: 

• Identify  the  relationships  that  exist  among  various  elements  and  with 
the  enemy. 

• Identify  alternative  courses  of  action,  appropriate  formation  maneuvers,  and 
fire  support  applications. 

• Communicate  decisions  to  subordinates  using  tactical  orders  so  that  decisions 
can  be  effectively  interpreted. 

• Manipulate  and  monitor  the  variety  of  tactical  radio  networks  available  at 
the  battalion  level. 


NAVY  TEAM  TRAINING 

As  in  the  other  Services,  team  training  in  the  Navy  consists  of  training  provided  to 
organized  teams  for  the  performance  of  a specific  mission.  The  Navy  discusses  team 
training  in  terms  of  five  categories:  Pre-Team  Indoctrination  Training,  Subsystem  Team 
Training,  System  Sub-Team  Training,  System-Level  Operational  Training,  and  Multi- 
Unit  System  Operational  Training  [26]. 

Pre-Team  Indoctrination  Training  is  conducted  in  a team  context  with  emphasis  on 
increasing  the  skill  levels  of  the  individuals  who  will  later  be  assigned  to  operational  teams. 

In  Subsystem  Team  Training  teams  vary  in  composition  under  different  conditions 
of  shipboard  readiness.  The  teams  are  divided  into  three  departments:  Combat  System 
Teams  (e.g.,  Search  Radar  Detection  and  Tracking),  Unit  Operations  Teams  (e.g.,  Navi- 
gation), and  Engineering  Systems  (e.g.,  Firefighting/Damage  Control). 

System  Sub-Team  Training  involves  the  training  of  two  or  more  subsystem  teams— 
generally,  an  entire  ship. 

System-Level  Operational  Training  is  accomplished  by  at-sea  training  because  existing 
pierside  facilities  are  not  capable  of  handling  an  entire  crew  of  a ship  in  all  facets  of  an 
operation.  This  type  of  training  is  best  described  as  a battle  problem  which  is  conducted 
by  underway  training  units  and  fleet  training  groups. 
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Multi-Unit  System  Operational  Training  is  in-port  exercises  utilizing  shore-based 
trainers  for  the  purpose  of  training  crews  prior  to  getting  under  way  for  the  exercise 
operations  area.  Major  in-port  exercises  utilize  the  Tactical  Advanced  Combat  Direction 
and  Electronic  Warfare  (TACDEW)  Trainer  [26] . 

The  following  major  team  training  devices  used  in  the  Navy  are  described  below: 

Device  2F87  (Weapons  Systems  Trainer),  Device  14A2  (Surface  Ship  Anti-Submarine  War- 
fare Early  Attack  Weapons  Systems  Trainer),  Device  14A6  (Anti-Submarine  Warfare  Coordi- 
nated Tactics  Trainer),  Device  21A37/4  (Submarine  Fleet  Ballistic  Missile  Training  Facility), 
and  TACDEW  [49], 

1.  Device  2F87,  Weapons  Systems  Trainer 

The  weapons  systems  trainer  (WST)  duplicates  the  interior  arrangement  and  appearance 
of  the  P-3C  aircraft.  Five  trainee  stations  simulate  corresponding  stations  in  the  aircraft. 

The  purpose  of  WST  training  is  to  teach  team  coordination.  Students  are  organized  into 
teams  according  to  positions  for  which  they  are  being  trained,  and  they  “fly”  simulated 
anti-submarine  warfare  (ASW)  missions.  These  missions  are  graduated  in  difficulty  from 
very  simple  scenarios  early  in  training  to  more  complex  exercises  toward  course  completion. 

2.  Device  14A2,  Surface  Ship  ASW  Early  Attack  Weapons  System  Trainer 

Device  14A2  is  used  to  train  surface  ship  teams  in  the  proper  utilization  of  opera- 
tional ASW  systems.  Training  emphasizes  the  procedural,  tactical  decision  making,  and 
team  coordination  activities  in  operating  and  employing  ASW  weapons  systems.  The  device 
provides  for  indoctrination  of  personnel  in  ASW  procedures  and  evaluation  of  tactical 
situations.  The  trainer  is  also  used  in  developing  and  planning  advanced  naval  undersea 
defense  tactics. 

The  trainer  occupies  over  3000  square  feet  of  floor  space  and  duplicates  the  physical 
configuration  of  major  operational  compartments  and  equipments  of  surface  ship  ASW 
attack  weapons.  It  simulates  their  functional  operation  and  responses,  such  as  target 
detection,  fire  control  solution,  and  weapon  launching  and  tracking. 

3.  Device  14A6,  ASW  Coordinated  Tactics  Trainer 

Device  14A6  is  designed  to  train  decision-making  personnel  in  the  tasks  they  must 
perform  when  engaged  in  coordinated  ASW  tactics.  Simultaneous  operation  of  48 
vehicles  of  various  types  and  a multiplicity  of  sensors  can  be  simulated.  Communications 
facilities  simulate  the  various  radio  channels  employed  operationally  to  coordinate  all 
phases  of  an  ASW  mission  from  search  through  attack.  Device  14A6  provides  a synthetic 
environment  within  which  ASW  personnel  can  practice  collecting  and  evaluating  ASW 
information,  making  decisions,  and  implementing  the  decisions  based  on  this  information. 
The  device  is  not  intended  to  train  equipment  operators;  therefore,  simulated  equipment 
does  not  resemble  fleet  equipment.  Functional  characteristics  of  the  simulated  equipment 
are  similar  to  fleet  equipment. 

Virtually  any  exercise  at  sea  which  requires  communication,  coordination,  maneuver- 
ing, and  decision  making  may  be  practiced  in  the  14A6  trainer  prior  to  going  to  sea. 

4.  Device  21A37/4,  Submarine  Fleet  Ballistic  Missile  (FBM)  Training  Facility 

Device  21A37/4  provides  training  in  offensive  and  defensive  tactics  for  nuclear  attack 
center  teams.  Surface  or  subsurface  maneuvers  may  be  accomplished,  and  training  may  be 
given  independently  or  in  coordination  with  other  units.  Instruction  of  senior  command 
and  staff  officers  in  direction  and  coordination  of  submarine  task  groups  with  surface  sup- 
port units  may  also  be  given. 
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A central  digital  computer  provides  problem  generation,  position,  and  motion  data 
generation.  Up  to  41  different  vehicles  can  be  included  in  training  problems.  A pro- 
jection system  in  the  attack  centers  permits  both  in-progress  monitoring  and  post-fire 
analysis  of  training  problems.  Attack  centers  can  be  operated  independently  or  operation 
can  be  coordinated  to  provide  submarine  versus  submarine  training.  Fifteen  different 
classifications  of  targets  are  currently  available,  12  at  any  one  time. 

5.  Tactical  Advanced  Combat  Direction  and  Electronic  Warfare  System  (TACDEW)1 

TACDEW  is  a highly  sophisticated,  computer-based  simulation  facility  having  three 
primary  missions:  individual  and  team  training,  tactics  evaluation,  and  testing  of  opera- 
tional computer  programs  for  Navy  Tactical  Data  System  (NTDS)  ships.  Training  is  con- 
ducted in  23  Combat  Information  Center  (CIC)  mockups  typical  of  the  ships  on  which 
trainees  serve  [49],  Team  and  multi-teair  training  is  accomplished.  The  purpose  of 
TACDEW  training  is  not  so  much  to  establish  team  interactive  skills  as  it  is  to  maintain 
or  enhance  these  skills  in  simulated  mission  contexts.  Training  typically  consists  of 
exercising  a given  team,  or  some  combination  of  teams,  in  a series  of  scenarios  of  graded 
difficulty.  The  scenarios  are  designed  to  model  tactical  situations  which  might  be 
encountered  in  an  operational  environment. 

MARINE  TEAM  TRAINING 

The  Marine  Corps  has  recently  initiated  two  tactical  exercise  projects  whose  objec- 
tives are  to  develop  and  integrate  computer-driven  systems  into  ongoing  field  and  school 
training  programs.  The  computer-driven  simulation,  which  is  the  heart  of  these  training 
systems,  can  handle  a greater  number  of  factors  with  greater  precision  and  finer  resolution 
than  was  previously  possible  in  any  manually  conducted  model  (83], 

The  two  team  training  systems  associated  with  these  projects  are  briefly  discussed 
below:  TESE  (Tactical  Exercise  Simulator  and  Evaluator);  TWAES  (Tactical  Warfare 
Analysis  and  Evaluation  System). 

i 

1.  TESE  (Tactical  Exercise  Simulator  and  Evaluator) 

TESE  is  used  to  train  Marine  Corps  officers  in  combat  decision  making.  The  project 
seeks  to  define  procedures  for  war  gaming.  The  goals  are  to  get  computer-based  meas- 
ures of  both  individuals  and  teams  during  amphibious  warfare  exercises,  and  to  increase 
the  number  of  trainees  who  can  be  processed. 

2.  TWAES  (Tactical  Warfare  Analysis  and  Evaluation  System) 

TWAES  is  a computer-assisted  system  to  control  tactical  field  training  exercises.  The 
system  offers  potential  improvements  in  maneuver  control  and  simulation  of  indirect 
fire  [26|.  The  current  TWAES  research  effort  is  studying  the  exact  role  that  the  post- 
exercise session  will  play  in  the  total  tactical  exercise.  The  question  being  addressed  is 
whether  this  feedback  session  will  be  a training  vehicle  to  further  extend  the  game’s 
learning  effectiveness  or  whether  it  will  be  merely  a post-exercise  debriefing  during  which 
administrative  information  about  game  procedures  is  passed  on  |83], 


1 A more  detailed  description  of  the  TACDEW  system  is  presented  in  the  TACDEW  Information 
Pamphlet  (34). 
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AIR  FORCE  TEAM  TRAINING 

In  the  Air  Force,  very  little  formal  team  training  is  being  conducted.  All  such  for- 
mal courses  surveyed  were  used  in  training  strategic  missile  teams.  Students  are  provided 
training  in  the  responsibilities  and  necessary  technical  procedures  of  the  appropriate  weap- 
ons system,  but  their  proficiency  upon  graduation  from  team  training  courses  is  le§s  than 
that  required  for  combat  readiness.  Thus,  additional  upgrade  training  is  required  at  the 
unit/base  prior  to  actual  job  performance  [26], 

Similarly,  there  are  few  team  training  devices.  One  device  identified  in  this  survey 
is  the  Functional  Integrated  Systems  Trainer  (FIST),  used  to  provide  a better  training  for 
the  fire  control  team  on  the  AC-130E  Gunship. 

Like  the  team  training  courses,  most  team  training  devices  deal  with  the  training  of 
aircraft  flight  crew  members.  Representative  training  devices  are  available  for  the  B-52, 
C-5,  C-130,  and  C-141  aircraft  as  described  below. 

1.  B-52  Weapons  System  Trainer 

The  WST  (G&H  models)  is  presently  under  development.  It  will  permit  the  inte- 
gration into  a single  team  training  device  of  four  individual  devices  presently  being  used: 
(a)  Mission  Trainer  used  for  training  pilots  and  co-pilots,  (b)  Navigator  Trainer  used  for 
training  radar  navigators  and  navigators,  (c)  Electronic  Warfare  Officer  (EWO)  Trainer,  and 
(d)  Gunnery  Trainer.  The  WST  when  developed  will  permit  simultaneous  team  training  of 
the  entire  six-man  B-52  crew.1 

2.  C-5  Mission  Flight  Simulator 

This  team  training  device  consists  of  three  training  stations  which  permit  integrated 
team  training  of  pilots/co-pilots,  navigator,  and  flight  engineer.2 

3.  C-130  Flight  Simulator  and  C-141  Flight  Simulator 

Both  of  these  training  devices  are  similar  to  the  C-5  Mission  Flight  Simulator  except 
that  training  is  not  provided  for  the  navigator  station.  However,  the  C-130  Flight  Simu- 
lator is  scheduled  to  add  the  navigator  station  as  part  of  the  device  in  1981. 3 

4.  Functional  Integrated  Systems  Trainer  (FIST) 

The  purpose  of  FIST  is  to  provide  a better  means  for  training  four  members  of  the 
fire  control  team  on  the  AC-130E  Gunship.  The  second  objective  of  FIST  is  to  refine 
and  promote  the  use  of  the  technology  for  developing  low-cost,  interlinked,  functional, 
part  task  trainers  [ 26  ] . 


SUMMATION 

The  emphasis  in  this  survey  has  been  on  formal  team  training.  In  addition,  much 
informal  team  on-the-job  training  occurs  in  the  daily  operations  of  the  Services.  In  such 
activities,  it  is  extremely  difficult  to  delineate  the  training  mode  from  the  operational, 
especially  in  areas  such  as  flight  training  and  air  defense. 

'Personal  communication  w th  COL  Roy  L.  Ripley  and  LTC  Walter  D.  Black  III,  Instructional 
Systems  Branch,  Audiovisual  and  Instructional  Systems  Division,  US  Air  Force,  Washington,  D.C. 

2 Personal  communication  with  LTC  C.R.  Philbrick,  MAC/DOTF,  Scott  Air  Force  Base,  IL. 

3 Ibid. 
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Various  methods  of  team  training  that  are  currently  under  way  in  the  Services  include 
the  following: 

• Large-scale  opposing  forces  tactical  exercises 

• Devices  which  simulate  effects  of  weapons 

• Computer  simulations  of  tactical  decision-making  problems 

• Simulations  of  mission-specific  hardware  and  software  (e.g.,  anti-submarine 
warfare  vehicles) 

These  methods  are  tailored  for  both  initial  team  training  and  maintenance  of  already  acquired 
team  skills.  They  vary  in  cost,  simulation  fidelity,  and  generalizability.  Although  these 
devices  are  now  being  used,  the  amount  of  programmatic  R&D  performed  upon  them  has 
been  relatively  small  considering  the  cost  of  such  systems. 


Chapter  5 

RESEARCH  IMPLICATIONS 


A number  of  implications  for  research  in  team  training  and  evaluation  are  presented 
in  this  chapter.  These  implications  were  drawn  from  a comparison  of  current  military 
practice  with  the  findings  of  the  literature  review.  The  experience  of  the  HumRRO  staff 
in  training  research  provided  additional  basis  for  identifying  the  areas  in  which  the  need 
for  support  is  most  critical.  The  suggested  programs  are  in  response  to  the  primary  ques- 
tions addressed  in  this  report: 

(1)  What  state-of-the-art  gaps  are  there  in  team  training  strategies  and 
evaluation  techniques'? 

(2)  What  new  team  training  and  evaluation  strategies  appear  to  hold  promise 
for  applications  to  the  DoD  environment  and  warrant  further  study? 

The  literature  reviewed  for  this  report  supports  the  contention  of  the  Defense  Science 
Board  [26]  that  there  is  a great  need  for  R&D  in  team  training,  considering  the  magnitude 
of  such  training  in  the  Services. 

One  major  problem  to  overcome  in  planning  research  in  this  area  is  the  tremendous 
diversity  of  terms,  concepts,  and  operations  in  the  ‘“earn  training.”  A “team”  is  not  a sim- 
ple fixed  unit;  usage  of  the  term  is  arbitrary,  has  unclear  boundaries,  and  does  not  denote 
stable  structure  and  composition.  A team  should  be  defined  on  the  basis  of  its  criterion 
situation,  not  on  the  basis  of  administrative  convenience  [42].  Describing  team  training 
in  the  Services,  the  Defense  Science  Board  [26]  resorted  to  an  awkward  abbreviation, 
CGTU,  for  Crew,  Group,  Team,  and  Unit  to  accommodate  users  of  each  of  these  terms. 
More  standardization  and  sharing  of  terms  would  contribute  to  more  effective  planning 
of  research  and  utilization  of  its  results.  Efforts  should  be  supported  to  develop  a reason- 
able taxonomy  upon  which  to  build  research  programs  in  team  training. 

A classification  scheme  proved  useful  for  categorizing  the  studies  and  techniques  dis- 
cussed in  this  review  along  two  dimensions— training  focus  and  task  situation.  The  train- 
ing focus  can  be  either  team  or  multi-individual,  depending  upon  the  skills  to  be  trained; 
the  task  situation  can  be  characterized  as  either  established  or  emergent. 

Methods  are  needed  for  differentiating  individual,  multi-individual,  and  team  skills. 
Techniques  need  to  be  developed  which  can  be  used  to  identify  individual  and  team 
skill  training  requirements  as  part  of  job/task  analyses.  Such  techniques  would  discrimi- 
nate between  isolated  and  interactive  behaviors  and  would  provide  the  opportunity  for 
establishing  criteria/standards  of  team  performance  as  part  of  training  requirement  anal- 
yses. Variations  and/or  combinations  of  observational  interaction  analyses  and  other 
unobtrusive  measures  which  have  been  used  in  non-classroom  settings  should  be  studied 
for  their  possible  application.  A recent  compendium  [90]  of  such  methods  provides 
detailed  information  about  techniques  which  are  of  potential  value,  such  as: 

• Interaction  Process  Analysis— Robert  F.  Bales 

• Sequential  Analysis  of  Verbal  Interaction  (SAVI)— Anita  Simon 

Yvonne  Agazarian 

• CERLI  Verbal  Behavior  Classification  (CVC)— Cooperative  Educational 

Research  Laboratory,  Inc. 

The  development  of  such  analytic  techniques  is  a critical  early  step  in  any  research  pro- 
gram that  seeks  to  use  realistic  team  performance  standards  as  effectiveness  criteria. 
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There  is  a considerable  need  for  measures  of  team  performance  to  apply  both  during 
and  following  training.  As  described  in  Chapter  3,  this  problem  has  resisted  solution  for 
the  past  20  years.  The  difficulties  of  observing  team  member  interactions,  the  great  costs 
in  money  and  time  to  conduct  studies  in  this  area,  and  the  unclear  relationships  between 
task  or  training  variables  and  team  performance  have  all  contributed  to  the  lack  of  suc- 
cess in  the  search  for  measures  of  team  proficiency.  The  production  of  standardizable, 
relatively  invariant  test  conditions  for  evaluating  dynamic  and  interactive  team  behavior 
remains  one  of  the  major  problems  in  team  evaluation. 

Only  in  recent  years  has  considerable  research  support  been  provided  to  seek  meas- 
ures of  team  performance.  Critical  problems  to  overcome  are: 

• The  criterion  problem— Without  agreed-upon  criteria  or  standards  of  team 
performance,  training  or  team  proficiency  cannot  be  evaluated. 

• The  measurement  problem  -What  to  measure,  where,  when,  and  how,  are  still 
all  unanswered  questions  in  the  evaluation  of  team  performance. 

Considering  the  current  lack  of  information  in  this  area,  the  potential  heuristic  value 
of  the  analytic  or  modeling  approach  is  worthy  of  continued  support.  Of  course,  the 
usefulness  of  any  model  depends  on  its  ability  to  generate  information  and  ideas  that  lead 
to  the  development  of  practical  and  worthwhile  evaluation  tools  and  techniques. 

For  example,  the  CARMONETTE  simulation  outputs  predictions  of  team  perform- 
ance based  on  hypothetical  values  for  a number  of  performance  and  situational  variables 
pertinent  to  the  team  and  to  an  opposing  force  (e.g.,  size/vulnerability,  firepower,  mobility, 
and  terrain  features).  Empirical  values  gathered  from  field  exercise  data  for  the  variables 
could  then  be  input  to  the  simulation  and  outcomes  compared.  Continued  systematic 
iterations  of  the  model  would  suggest  the  specific  variables  that  made  a difference.  If 
validated  in  further  field  exercises,  these  variables  could  guide  the  refinement  of  test  situa- 
tions and  the  development  of  better  measurement  techniques. 

It  is  probably  true  that  teams  operate  in  situations  that  have  a combination  of  both 
emergent  and  established  characteristics.  This  underlies  the  importance  of  performing  a 
thorough  analysis  of  the  criterion  task  situation  prior  to  developing  team  training  pro- 
grams. Applying  the  systems  approach  to  the  development  of  team  training  should  permit 
identification  of  the  interactions,  communication,  coordination,  decision  making,  and 
o',  her  activities  required  in  the  task  performance  of  each  team  member.  The  information 
gathered  with  the  appropriate  analytical  tools  should  include  standards  and  relevant  con- 
ditions of  performance.  This  would  be  a major  step  toward  obtaining  clearly  stated, 
objective  criteria  and  procedures  for  evaluating  team  performance. 

One  consistent  finding  of  this  literature  review  was  that  individual  proficiency  has 
been  shown  to  be  a major  determiner  of  team  performance.  This  had  led  investigators 
such  as  Hall  and  Rizzo  [49]  to  conclude  that  more  emphasis  should  be  placed  upon 
individual  rather  than  team  training.  The  authors  of  this  review  do  not  agree.  Although 
certain  laboratory  studies  showed  that  team  training  was  superfluous  when  the  tasks 
required  only  individual  skills,  other  investigations,  primarily  in  more  realistic,  “emergent” 
situations,  showed  the  importance  of  team  training.  When  “team  skills”  (e.g.,  inter- 
action), even  though  poorly  defined,  were  important  in  the  task  situation,  team  training 
was  more  effective  than  individual  training. 

Further  research  into  this  area  should  provide  information  which  can  impact  on  the 
issue  of  where  and  when  to  conduct  team  training.  For  example,  individuals  in  air  squad- 
rons are  trained  within  a team  context  and  are  then  individually  assigned  to  operational 
units.  If  team  skills  are  relevant  to  a given  task  situation  and  can  be  learned  during  a 
limited  training  period,  it  may  be  more  efficient  to  assign  the  team  intact  rather  than 
disband  it  following  training  [49], 

A need  exists  for  studies  which  explore  the  issue  of  skill  training  sequencing.  After 
individual  and  team  skill  requirements  for  a given  task  situation  have  been  identified,  the 
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sequence  in  which  component  skills  are  trained  can  be  systematically  varied  and  the 
resulting  effects  compared.  For  example,  individual  skills  can  be  trained  prior  to  team 
skills  training;  training  of  team  skills  can  occur  first;  or  both  individual  and  team  skills 
can  be  integrated  within  the  training  program  and  taught  concurrently.  The  results  of 
such  studies,  which  can  be  performed  in  either  established  or  emergent  task  situations, 
will  help  determine  when  and  under  what  conditions  team  training  can  be  administered 
most  effectively. 

Some  types  of  team  training  are  so  expensive  that  one  cannot  conduct  them  in  an 
operational  environment.  Other  types  of  training  have  logistic  or  safety  factors  that  pre- 
clude operational,  full-fidelity  training  or  testing.  High-fidelity  simulations  which  have 
been  common  in  individual  training  for  many  years  have  been  shown  to  be  effective  for 
team  training.  However,  such  simulation  techniques,  though  costly,  provide  little  generality 
beyond  a specific  training  mission.  A substantial  research  and  development  effort  is  needed 
to  determine  the  degree  of  simulation  fidelity  required  for  effective  training  in  different 
situations.  While  relatively  low-fidelity  simulation  techniques  have  been  effective  and 
economical  in  individual  training,  the  review  of  the  literature  indicated  that  high-fidelity 
simulations  were  needed  for  effective  team  training.  However,  the  studies  surveyed  were 
performed  in  established  laboratory  settings.  The  search  should  continue  for  effective  low- 
fidelity  devices  or  methods  to  apply  to  team  training  in  emergent  task  situations  because 
of  their  potential  cost  savings. 

The  second  question  addressed  in  this  review  involved  the  identification  of  team 
training/evaluation  techniques  which  warranted  further  study  and  support.  One  of  the 
most  promising  new  approaches  is  two-sided  engagement  training.  Current  implementa- 
tions of  this  technique  were  described  in  Chapter  4 in  discussions  of  SCOPES,  REALTRA1N, 
and  MILES.  The  two-sided  engagement  simulation  provides  a low-cost,  high-fidelity  frame- 
work for  developing  individual  skills.  It  provides  a situation  in  which  there  is  high  moti- 
vation fostered  by  competition  in  a natural,  realistic  environment.  This  approach,  which 
can  be  classified  as  situational  or  contextual  training,  is  an  outgrowth  of  the  effective 
“functional  context”  approach  to  individual  training  [85], 

Up  to  now,  two-sided  engagement  training  has  been  used  to  teach  individual  skills 
within  a group  context.  Support  is  needed  to  study  the  feasibility  of  applying  these 
methods  to  train  team  skills.  There  is  a need  to  study  how  effective  the  techniques  are, 
analyze  the  factors  which  produce  favorable  training  outcomes,  and  explore  how  these 
methods  can  be  transferred  to  other  training  situations.  Thus  far,  post-exercise  debrief- 
ings have  been  used  to  discuss  individual  performances.  Team  performance  information 
needs  to  be  fed  back  to  trainees  in  a more  timely  and  effective  manner  both  during  and 
after  training. 

A systematic  investigation  of  feedback  effects  upon  team  training  is  needed.  Sup- 
port for  research  on  the  following  issues  would  be  particularly  useful: 

• Need  for  feedback— What  changes  in  training  methods  or  content  result  from 
feedback?  Will  there  be  differences  if  remediation  is  contingent  or  non- 
contingent upon  feedback?  , 

• Feedback  mode— Methods  should  be  available  for  giving  both  trainees  and 
trainer  feedback  of  how  well  the  team  is  performing.  Identifying  appropri- 
ate methods  for  giving  performance  feedback  to  team  members  is  an  area 
requiring  further  R&D  (e.g.,  compare  video  recording  with  post-exercise 
debriefing  procedures— to  test  the  hypothesis  that  clarity  and  objectivity  of 
observation  will  provide  more  useful  feedback).  The  use  of  television  and 
video  recording  to  provide  relatively  responsive  performance  feedback  has 
been  used  successfully  in  a variety  of  contexts  (e.g.,  sports,  teacher  training). 

Its  potential  value  to  military  team  training  appears  great,  but  considerable 
R&D  support  is  needed  to  identify  its  most  appropriate  applications. 
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• Delay  of  feedback— A systematic  study  is  needed  to  determine  the  conditions 
under  which  provision  of  feedback  immediately  after  a training  exercise  is 
superior,  equal  to,  or  inferior  compared  to  delayed  presentations  (1,  2,  or 
more  days  following  the  event). 

Another  issue  which  merits  study  concerns  the  effects  of  team  member  familiarity 
on  the  acquisition  of  team  skills.  Does  the  length  of  time  that  a team  functions  together 
affect  team  performance?  One  could  systematically  vary  this  time  parameter,  re-allocate 
team  members,  set  up  new  teams,  and  then  study  transfer  effects. 

The  ability  to  evaluate  oneself  accurately  is  a skill  that  should  help  to  produce  effec- 
tive team  performance.  Analysis  of  one’s  own  errors,  and  awareness  that  one  is  over- 
burdened and  needs  other  team  members’  help  are  two  specific  components  of  this  abil- 
ity which  should  be  trained.  There  is  a need  for  a program  supporting  the  development 
and  evaluation  of  training  packages  which  would  provide  generalizable  instruction  in 
these  self-assessment  skills.  One  could  compare  subsequent  team  performance  with  or 
without  prior  self-assessment  training.  Greater  proficiency  would  be  expected  from  those 
teams  who  have  had  such  self-assessment  skill  training. 

Another  possibility  is  that  prior  training  of  team  members  to  assess  the  capabilities  of 
others  will  result  in  better  subsequent  team  performance.  Having  the  ability  to  assess  other 
team  members’  strengths  and  weaknesses  might  provide  the  basis  for  selection  of  the 
individual  team  member  best  suited  to  take  over  or  help  out  on  a given  activity— and 
thereby  enhance  overall  team  performance. 

It  may  be  more  efficient  to  do  research  on  a combination  of  some  of  the  issues  dis- 
cussed above  within  the  same  study.  For  example,  group  feedback  or  skill  sequencing 
issues  can  be  studied  in  environments  that  contain  established  or  emergent  task  situations 
and  that  vary  in  their  degree  of  simulation  fidelity. 

The  research  and  development  implications  mentioned  in  the  preceding  paragraphs 
are  merely  major  examples  of  what  should  be  considered  programmatic  efforts  in  the 
team  training/evaluation  area.  Considering  the  importance  and  extent  of  this  area,  the 
current  state  of  knowledge  is  quite  meager.  Studies  and  techniques  such  as  those  men- 
tioned, if  supported,  should  help  to  remedy  this  condition. 
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